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Introduction 

Currently,  technology  has  evolved  to  potentially  transform  learning  systems,  however;  to  achieve 
effective  learning  tools,  great  effort  must  be  taken  to  develop,  test  and  disseminate  the 
technology.(3)  A  systematic,  reproducible  approach  to  research  and  development  must  be 
implemented  so  that  results  produce  usable,  validated,  learning  systems.  SimCenter  Hawaii’s 
research  model  Technology-Enabled  Learning  and  Intervention  Systems  (TELIS)  ( Figure  1 ) 
represents  the  overarching  theme  of  advanced  health  care  technologies  and  educational  research 
being  undertaken  by  our  interdisciplinary  research  group  in  Hawaii.  TELIS  is  motivated  by  The 
Learning  Federation  Project’s  technology  research  roadmaps,  which  provide  an  important  needs 
assessment  for  current  and  future  TELIS  applications.(l-3)  SimCenter  Hawaii  has  developed  a 
model  that  is  based  on  three  related  action  areas:  development,  application,  and  measurement. 


Figure  1.  SimCenter  Hawaii:  Research  and  Development  Model  for  TELIS  Applications 
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Despite  the  advantages  of  virtual  reality  (VR)  medical  simulations,  due  to  cost  and  availability, 
the  educational  technologies  used  for  training  remain  significantly  underdeveloped  and 
underutilized.(lO)  Simulation  technology  has  not  become  part  of  the  core  curriculum  at  most 
medical  schools. (8)  In  the  federal  arena  as  of  2005,  only  14  out  of  198  federal  training  courses 
on  terrorism  used  computer/technology-based  learning  applications;  only  6%  of  the  training 
systems  were  VR  based,  and  only  31%  were  simulation  based.(15) 


Page  4  of  86 


W81XWH-07-2-0062  12/12/2008 

SimCenter  Hawaii:  Virtual  Reality  Applications  for  Health  Care  Education  and  Training 


SimCenter  Hawaii  Projects.  Projects  in  the  SimCenter  Hawaii’s  model  may  apply  diverse 
technologies  for  diverse  topics,  but  sustain  the  common  goal  of  technological  innovation, 
research,  and  development,  in  simulation  technology  with  the  desired  outcome  of  producing  and 
validating  advanced  learning  systems  and  methodologies.  The  following  projects  are  proposed 
under  the  SimCenter  Hawaii’s  TELIS  model: 

•  Project  i,  VR  Nephron  -  Development:  Renal  physiology  contains  many  abstract  concepts 
that  have  been  identified  as  difficult  for  students  to  learn  and  understand,  with  the  counter- 
current  exchange  system  ranking  number  one  in  basic  science  difficulty  and  number  two  in 
clinical  difficulty.(6)  It  is  postulated  that  by  visualizing  and  interacting  with  these  difficult 
concepts,  student  learning  will  be  enhanced. 

This  project  will  provide  further  development  for  the  VR  Nephron  application  —  a 
previously  developed  renal  physiology  prototype  —  by  adding  an  interactive  gaming 
component,  which  allows  users  to  navigate,  explore  and  interact  with  the  3D  environment.  A 
new  rules-based  engine  will  be  developed  providing  intelligence  to  the  application. 

Graphical  models  and  the  user  interface  will  be  refined  and  designed  to  teach  the  counter- 
current  mechanism  in  renal  physiology. 

After  the  initial  prototype  is  complete,  heuristic  usability  studies  combined  with  a  user 
evaluation  will  be  conducted  to  improve  software  design  and  assess  the  usability  of  the  VR 
Nephron  application.  This  incremental  development  process  permits  early  design 
modification,  which  saves  both  time  and  money,  instead  of  having  to  redesign  a  system  after 
extensive  work  has  already  been  completed.(14)  After  prototype  completion,  future  studies 
will  determine  whether  this  tool  enhances  the  learning  of  renal  physiology. 

•  Project  2,  VR  Triage  -  Application  and  Measurement:  VR  is  a  TELIS  with  significant 
potential  for  triage  training.  A  2003  study  by  the  Federation  of  American  Scientists 
concluded  that  the  U.S.  has  a  large,  but  unmet  need  to  train  first  responders. (10)  VR 
applications  have  theoretical  advantages  over  paper  exercises,  actors,  and  even  manikin 
simulations  due  to  scenario  flexibility,  setup  and  space  requirements,  real-time  feedback, 
integrated  evaluation,  and  portability.  The  virtual  environment  can  be  designed  to  mimic 
battlefield  or  mass  casualty  settings,  and  trainees  can  go  through  the  steps  of  triage  and  be 
tested  by  the  computer.  Although  certain  VR  systems  can  provide  realistic,  procedural  skills 
training,(12)  the  scenarios  developed  for  triage  would  primarily  develop  and  assess  cognitive 
skill  sets.  As  such,  a  VR  triage  curriculum  would  be  an  excellent  method  to  develop,  assess, 
and  provide  prospective  training  for  providers  like  emergency  medical  technicians,  for  whom 
practice  of  basic  patient  stabilization  procedures  would  be  redundant.  It  is  more  important 
that  these  professionals  practice  their  decision-making  skills  in  a  mass  casualty  scenario,  or 
some  other  uncommon  occurrence.  With  this  in  mind,  the  VR  training  platform  for  triage 
could  be  used  for  several  purposes:  a)  initial  training  for  those  without  any  experience  as 
prelude  to  manikin  training;  b)  refresher  training  between  intervals  of  manikin  training;  and 
c)  competency  assessment  of  higher  level  decision-making  skills. 

Overall  for  the  Virtual  Triage  project,  we  propose  that  VR  can  be  utilized  in  triage 
training  for  several  purposes:  a)  initial  and  subsequent  training  (with  or  without  previous  VR 
training)  for  decision-making  skills,  and  b)  competency  assessment  of  decision-making  triage 
skills.  Through  previous  funding,  a  VR  triage  application  has  been  developed.  The  current 
proposal  conducts  a  user  study  to  measure  educational  outcomes.  The  intention  of  the 
project  is  to  improve  learning  and  performance  through  VR  training.  Students  will  pre-train 
using  a  web-based  training  module,  and  then  conduct  triage  on  three  sequential  triage 
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scenarios,  each  with  five  VR  casualties.  Self-efficacy  will  be  assessed  through 
questionnaires  before  and  after  the  VR  training. 

•  Project  3,  Virtual  Reality  Motor-Skills  Trainer  (VRMSS}  -  Application  and  Measurement: 
With  the  Virtual  Reality  Motor-Skills  Simulator  (VRMSS),1 (5)  a  TELIS  has  been  previously 
developed  to  train  users  on  baseline  fine-motor  skills  used  in  surgery  in  an  abstract  three 
dimensional  environment.  Key  features  include  hand  training  for  dominant  and  non¬ 
dominant  hands,  and  hands  working  in  tandem.  Haptics  produces  force  feed-back,  which 
provides  touch  and  feel  to  the  application  for  greater  realism.  In  order  to  demonstrate  the 
learning  benefit  of  VRMSS,  this  study  will  measure  performance  outcomes  on  a  previously 
validated  commercial  simulation  trainer  between  VRMSS  and  non- VRMSS  trained  groups. 

Minimally  invasive  surgery  has  been  shown  to  have  advantages  over  conventional  open 
methods.  Laparoscopic  procedures  now  represent  the  ‘gold  standard’  for  various  surgical 
procedures.  However,  training  is  necessary  to  ensure  a  high-quality  treatment  of 
patients.(13)  Surgeons  are  currently  trained  using  conventional  box  trainers  as  well  as  high- 
cost  virtual  reality  simulators.  VR  simulation  has  been  validated  as  an  effective  training 
method  for  laparoscopy, (9,1 1)  and  has  several  advantages  over  box  trainers  including 
automated  scoring.  The  Virtual  Reality  Motor-Skills  Simulator  (VRMSS),  with  low-cost  3D 
haptics  input  devices,  is  targeted  for  the  medical  student  and  intern  level.  Such  a  system  can 
be  implemented  at  the  fraction  of  the  cost  of  commercial  counterparts.  VRMSS  has  the 
special  ability  to  run  from  a  distributed  network  location,  enabling  distance  training  and 
improving  access  to  this  type  of  education. 

See  Appendix  D  -  For  ppt  presentation  on  overview  of  projects 
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Body 

Task  A.  Project  1,  VirtiuU  Reality  Nephron  -  Development 

This  study  describes  the  development  of  a  multi-agent  system  to  simulate  kidney  function  for  the 
purpose  of  teaching  renal  physiology  to  healthcare  students.  Renal  function  is  modeled  with 
particles  that  move  within  an  environment,  using  the  agent  paradigm.  Particles  represent 
molecules  and  fluids  and  the  environment  represents  the  structures,  membranes  and  volumes  of 
the  kidneys.  Particles  move  dynamically  through  the  system,  responding  appropriately  depending 
on  their  surroundings.  Results  of  heuristic  and  usability  testing  by  medical  students  demonstrated 
that  the  3D  virtual  reality  educational  tool  developed  was  well  accepted  by  users. 

Study  Design 

The  studies  took  place  at  a  medical  school  simulation  center.  Data  for  the  heuristic  evaluation 
and  the  individual  user  evaluation  were  collected  through  interviews  and  written  surveys.  The 
research  protocols  were  approved  by  the  University  of  Hawaii  Committee  on  Human  Studies  and 
the  United  States  Army  Medical  Materiel  and  Research  Command  Office  of  Research 
Protection. 

Heuristic  Evaluation 

Usability  testing  is  critical  to  the  development  of  an  effective  software  application  to  ensure  that 
the  learner  is  able  to  focus  on  the  educational  objective  rather  than  the  process  of  completing 
tasks  in  the  virtual  environment.  The  heuristic  usability  evaluation  is  a  systematic  approach  to 
software  development  that  is  designed  to  identify  significant  usability  issues  early  in  the  software 
development  life  cycle.  The  heuristic  evaluation  of  this  application  was  modeled  on  the  method 
proposed  by  Nielsen  (1993)  for  software  usability  testing.  Each  new  evaluator  contributes 
progressively  less  new  information  regarding  usability  issues  and  problems.  By  the  fifth 
usability  tester,  85%  of  problems  have  been  identified,  and  additional  testers  mostly  replicate 
information  that  has  already  been  discovered.  Tang  and  colleagues  (Tang,  2006)  found  that  a 
second  iteration  of  the  heuristic  methodology  demonstrated  additional  improvement  in  the  user 
interface  that  was  being  developed. 

In  this  study,  a  convenience  sample  of  medical  students  and  one  faculty  member  was  recruited 
by  email.  Two  sessions  were  planned  separated  by  approximately  one  month  of  software 
development.  Open-ended  comments  by  the  evaluators  were  collated  and  categorized  as  gaming 
or  content  issues. 

Individual  User  Evaluation 

A  convenience  sample  of  medical  students  was  recruited  by  email.  Users  were  given  a  group  of 
seven  tasks  to  perform  before  completing  a  survey.  The  survey  consisted  of  five  domains,  each 
rated  on  a  7-point  Likert  scale  from  1  unacceptable  to  7=exceeds  expectations.  An  acceptable 
level  was  defined  as  a  score  >  4.  The  domains  were  leamability,  efficiency,  memorability,  errors, 
and  satisfaction.  The  task  list  for  users  was  this: 

1 .  Login  to  the  virtual  nephron  game; 
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2.  Orient  to  the  environment:  within  the  first  person  view,  change  the  orientation  to  identify 
proximal,  distal,  cortical,  and  medullary  perspectives; 

3.  Use  the  paddle  interface  to  find  and  select  the  glomerulus,  proximal  convoluted  tubule, 
and  collecting  duct. 

4.  Use  the  paddle  interface  to  change  the  view  from  hovering  view  to  head-up  view,  and 
from  hovering  view  to  foot  view; 

5.  Use  the  paddle  interface  to  change  the  functional  representation  of  the  nephron,  and  to 
activate  sodium  particles  and  urea  particles; 

6.  Use  the  paddle  interface  to  navigate  to  different  sites  of  active  transport  (depicted  in  the 
mechanical  view  as  moving  pistons 

7.  Exit  the  virtual  environment. 

See  Appendix  A  -  Wang  K,  Reed  N,  Vincent  D.  (2009).  Multi-Agent  Simulation  of  Kidney 
Function.  In  Multi-Agent  Systems  for  Health  Care  Simulation  and  Modeling.  IGI  Global  (in 
press). 

Task  B.  Project  2,  Virtual  Reality  Triage  -  Application  and  Measurement 

Virtual  reality  (VR)  environments  offer  potential  advantages  over  traditional  paper  methods, 
manikin  simulation,  and  live  drills  for  mass  casualty  training  and  assessment.  We  measured  the 
acquisition  of  triage  skills  by  novice  learners  after  exposing  them  to  three  sequential  scenarios 
(A,  B,  and  C)  of  five  simulated  patients  each  in  a  fully  immersed  3D  VR  environment.  We 
hypothesized  that  learners  would  improve  in  speed,  accuracy,  and  self-efficacy. 

Methods.  Twenty-four  medical  students  were  taught  principles  of  mass  casualty  triage  using 
three  short  podcasts,  followed  by  an  immersive  VR  exercise  in  which  learners  donned  a  head 
mounted  display  and  three  motion  tracking  sensors,  one  for  their  head  and  one  for  each  hand. 
They  used  a  gesture-based  command  system  to  interact  with  multiple  VR  casualties.  For  Triage 
Score,  one  point  was  awarded  for  each  correctly  identified  main  problem,  required  intervention, 
and  triage  category.  For  Intervention  Score,  one  point  was  awarded  for  each  correct  VR 
intervention.  Before  and  after  surveys  measured  self-efficacy  and  reaction  to  the  training. 

Results.  Four  students  were  excluded  from  analysis  due  to  participation  in  a  recent  triage 
research  program.  Results  from  twenty  students  were  analyzed.  Triage  Scores  and  Intervention 
Scores  improved  significantly  during  scenario  B  (p<0.001).  Time  to  complete  each  scenario 
decreased  significantly  from  A  (8:10  minutes)  to  B  (5.15  minutes)  (p<0.001)  and  from  B  to  C 
(3:58  minutes)  (p<0.001).  Self-efficacy  improved  significantly  in  the  areas  of  prioritizing 
treatment,  prioritizing  resources,  identifying  high  risk  patients,  and  beliefs  about  learning  to  be 
an  effective  first  responder. 


Medical  student 
year  of  training 

No. 

(%) 

MSI 

12(60) 

MS2 

3(15) 

MS3 

3(15) 

MS4 

2(10) 
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Table  1.  Medical  school  year  of  training 


VR 

Simulation 
Mean  (SD) 

1 .  The  material  covered  was  relevant  to  my  duties  as  a  healthcare 
team  member. 

6.5(0.61) 

2.  The  course  objectives  were  adequately  explained. 

6.3(0.97) 

3.  The  course  was  well  organized. 

6.5(0.69) 

4.  The  material  was  presented  in  an  interesting  way. 

6.8(0.55) 

5.  The  course  communicated  the  material  effectively. 

6.7(0.59) 

6.  As  the  course  progressed,  my  questions  were  answered. 

6.8(0.44) 

Table  2.  Evaluation  of  VR  triage  course  (7-point  Likert  scale:  l=strongly  disagree,  4=neutral, 
7=strongly  agree) 


VR 

Simulation 
Mean  (SD) 

1.  How  was  the  pace  of  the  course?  (l=too  slow,  4=just  right, 

7=too  fast) 

4.2(0.39) 

2.  How  was  the  level  of  difficulty  of  the  course?  (l=too  hard  to 
understand,  4=just  right,  7=too  easy) 

4.5(0.83) 

Table  3.  Pace  and  difficulty  of  VR  triage  course 


Before 

Mean 

(SD) 

After 

Mean 

(SD) 

P 

value 

1 . 1  feel  confident  that  I  will  learn  to  be  an  effective  first  responder. 

4.0(0.69) 

4.2(0.62) 

0.034 

2. 1  feel  confident  that  patients  will  consider  me  an  effective  first 
responder. 

3.8(0.64) 

4.1(0.60) 

0.006 

3. 1  feel  confident  in  my  ability  to  prioritize  the  treatment  of  patients 
in  a  mass  casualty  situation. 

3.2(0.89) 

4.2(0.52) 

0.001 

4. 1  feel  confident  in  my  ability  to  prioritize  the  use  of  resources  in  a 
mass  casualty  situation. 

3.1(0.97) 

4.2(0.75) 

0.001 

5. 1  feel  confident  in  my  ability  to  identify  high  risk  patients  for 
immediate  treatment  in  a  mass  casualty  situation. 

3.4(.88) 

4.2(0.77) 

0.008 

Table  4.  Self-efficacy  (5-point  Likert  scale:  l=never  to  5=always) 
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Figure  1.  A  medical  student,  wearing  a  head  mounted  display  and  two  sensor-gloves,  checks  the 
pulse  of  a  simulated  casualty  in  the  virtual  environment. 


Figure  2.  The  same  student  checks  a  simulated  casualty’s  carotid  pulse  in  the  virtual 
environment,  using  a  virtual  hand  that  is  mapped  to  their  right  sensor-glove.  The  fingers  of  the 
virtual  hand  move  up  and  down  at  the  pulse  rate  when  positioned  over  the  carotid,  radial,  or 
femoral  regions.  Pulse  rate  and  strength  also  appear  in  the  left  upper  visual  field  of  the  head 
mounted  display.  Some  students  described  “feeling”  the  pulse  of  the  VR  patient. 
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Figure  3.  Study  design.  RQ  =  Reaction  Questionnaire,  LEQ  =  Learner  Evaluation  Questionnaire 


ABC 

Scenario 


Figure  4.  Average  Triage  Score  per  learner  (max=15).  Significant  difference,  *p  <  0.001 
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Figure  5.  Average  Intervention  Score  per  learner  (max=5).  Significant  difference,  *p  <  0.001 


Figure  6.  Time  to  triage  one  scenario  consisting  of  five  simulated  patients.  Significant  difference, 
*p<0.001,  **p<0.05 

See  Appendix  B  -  Vincent  D,  Sherstyuk  A,  Burgess,  Connolly  K.  Teaching  Mass  Casualty 
Triage  Skills  using  Immersive  Three-dimensional  Virtual  Reality.  Acad  Emer  Med  2008:  15:1-6. 
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Task  C.  Project  3,  Virtual  Reality  Motor-Skills  Trainer  -  Application  and_  Measurement 

A  prototype  low-cost  virtual-reality  motor-skills  simulator  (VRMSS)  was  created  using  the 
SPRING  platform  as  the  underlying  development  tool  at  the  Telehealth  Research  Institute,  John 
A.  Bums  School  of  Medicine,  in  conjunction  with  the  National  Biocomputation  Center,  Stanford 
University.  The  VRMSS  was  specifically  designed  to  teach  baseline  fine-motor  skills  used  in 
surgery  that  are  based  on  a  matrix  of  elemental  technical  skills  that  comprise  the  tenets  of 
surgical  technique.  Following  initial  audio/video  based  instruction;  bead-like  objects  are  moved 
to  holding  cups/target  areas  within  a  non-threatening  abstract  3-Dimensional  environment  in 
laparoscopic,  microscopic,  and  endoscopic  formats. 

Methods 

Fifty-seven  participants  were  randomly  assigned  to  one  of  three  groups  to  conduct  an  evaluation 
study  of  the  VRMSS.  To  assess  fine-motor  skills  participants  performed  the  basic  skills  of 
grasping,  lift  &  grasping  and  coordination  on  the  LapSim  from  Surgical  Sciences.  Each  group 
then  received  time-equivalent  training.  The  first  group  trained  on  the  VRMSS.  The  second 
group  trained  on  a  box  trainer  used  for  laparoscopic  training.  The  third,  control  group,  read 
through  two  chapters  of  online  curriculum  on  telemedicine.  Participants  were  then  reassessed  on 
the  LapSim  for  changes  in  fine-motor  surgical  skills. 

Results 

The  grasping  and  lifting  &  grasping  tasks  were  not  significantly  difficult  and  all  groups 
performed  well  on  those  tasks.  Figures  4  and  5  show  highlights  of  group  performance  on  the 
LapSim  coordination  task  after  training.  Group  performance  times  in  Figure  4  show  a  significant 
difference  between  the  groups  (mean  =  66.3  seconds,  F=3.423,  p<0.04).  The  post  hoc  test 
(Tukey)  showed  that  VRMSS  and  the  box  trainer  mean  completion  times  were  not  significantly 
different  and  the  control  group  had  a  significantly  longer  performance  time  (p<0.044).  Figure  5 
shows  the  pass  percentage  for  the  groups.  The  VRMSS  and  box  trainer  groups  were  very  similar 
and  both  training  were  at  least  1 .75  times  better  than  the  control  group. 
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LapSim  -  PerformanceTime 


Figure  1.  Shown  is  the  mean  time  of  each  group  on  the  coordination  task  after 
receiving  training 


LapSim  -  Passes  the  T est 


Figure  2.  Shown  is  the  percent  of  participants  from  each  group  that  passed  the 
LapSim  on  the  coordination  task  after  receiving  training. 


See  Appendix  C  -  Ikehara  C,  Aschwanden  C,  Burgess  L,  Montgomery  K,  Mok  D.  (2009). 
Evaluating  a  Virtual  Reality  Motor-Skills  Simulator.  MMVR  2009. 
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Key  Research  Accomplishments 

TaskA.  Project  7,  Virtual  Reality.  Nephron  -  Development 

Task  Al.  Develop  interactive  gaming  virtual  reality  nephron  application  -  A  user  interface  was 
created  for  selecting  and  deselecting  key  features  of  the  model,  enabling  user  defined 
educational  exploration  to  take  place. 

Task  A2.  Complete  all  appropriate  procedures  with  institutional  review  boards  -  Protocol  has 
been  approved  by  USAMRMC  ORP  HRPO  for  exempt  status  (in  addition  to  UH 
IRB).  All  appropriate  procedures  with  institutional  review  boards  have  been 
completed. 

Task  A3.  Conduct  heuristic  usability  study  and  present/draft  results  -  see  appendix  A 


Task  B.  Project  2,  Virtual  Reality  Triage  -  Application  and  Measurement 


Task  Bl. 


Task  B2. 
Task  B3. 


Task  B4. 


Complete  all  appropriate  procedures  with  institutional  review  boards  -  Protocol  has 
been  approved  by  USAMRMC  ORP  HRPO  for  exempt  status  (in  addition  to  UH 
IRB).  All  appropriate  procedures  with  institutional  review  boards  have  been 
completed. 

Conduct  acquisition  of  triage  skills  study  -  24  participants  successfully  completed  the 
study. 

Skills  acquisition  relative  to  self-efficacy  study  -  results  showed  that  self-efficacy 
improved  significantly  in  the  areas  of  prioritizing  treatment,  prioritizing  resources, 
identifying  high  risk  patients,  and  beliefs  about  learning  to  be  an  effective  first 
responder. 

Analyze  data,  interpret  results  and  draft  manuscript  for  publication  -  see  Appendix  B 


Task  C.  Project  3,  Virtual  Reality  Motor-Skills  Trainer  -  Application  and  Measurement 

Task  Cl.  Complete  development  of  VRMSS  trainer  -  VRMSS  software  and  scenarios  were 
developed  and  tested. 

Task  C2.  Complete  all  appropriate  procedures  with  institutional  review  boards;  prepare  flyers 

for  participant  recruitment;  setup  equipment  and  tasks  -  Protocol  has  been  approved  by 
USAMRMC  ORP  HRPO  for  exempt  status  (in  addition  to  UH  IRB).  All  appropriate 
procedures  with  institutional  review  boards  have  been  completed. 

Task  C3.  Conduct  acquisition  of  surgical  skills  study  -  60  participants  have  undergone  the 

surgical  skills  study.  Data  including  performance  measures  and  questionnaires  has 
been  collected  from  all  60  participants. 

Task  C4.  Analyze  data,  interpret  results,  and  draft  manuscript  for  publication,  and  update  web  site 
-  see  Appendix  C 
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Reportable  Outcomes 

•  Publication  -  Wang  K,  Reed  N,  Vincent  D.  (2009).  Multi-Agent  Simulation  of  Kidney 
Function.  In  Multi-Agent  Systems  for  Health  Care  Simulation  and  Modeling.  IGI 
Global.  (Appendix  A) 

•  Publication  -  Vincent  D,  Sherstyuk  A,  Burgess,  Connolly  K.  Teaching  Mass  Casualty 
Triage  Skills  using  Immersive  Three-dimensional  Virtual  Reality.  Acad  Emer  Med  2008 
15:1-6. 

•  Publication  -  Ikehara  C,  Aschwanden  C,  Burgess  L,  Montgomery  K,  Mok  D.  (2009). 
Evaluating  a  Virtual  Reality  Motor-Skills  Simulator.  MMVR  2009  (Appendix  C) 
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Conclusions 

Simulator-based  training  has  been  shown  to  improve  outcomes  for  both  cognitive  as  well  as 
motor-skills  training.(7)  Cognitive  modules  can  be  distributed  through  advanced  learning 
networks.  (4)  This  has  significant  implications,  because  enterprise  wide  educational  solutions 
can  be  developed  at  one  center  and  accessed  by  the  entire  enterprise  remotely.  The  Army 
Knowledge  Network  is  an  example  of  such  a  solution,  and  sophisticated  training  modules  being 
developed  at  SimCenterHawaii  can  be  placed  centrally  and  accessed  remotely.  In  addition  to  the 
learning  networks,  experiential  training  with  virtual  reality  can  better  meet  training  objectives  by 
simulating  the  “real  patient.”  A  distributed  curriculum,  which  can  provide  greater  accessibility 
to  didactic  learning  materials,  coupled  with  a  hands-on  VR-based  and  virtual  reality  training 
curriculum  can  provide  a  multidisciplinary  training  program  to  enhance  the  training  of  first 
responders. 

Our  research  within  this  proposal  strives  to  further  develop  and  demonstrate  that  simulation 
training  with  virtual  reality  software  and/or  manikins  can  not  only  better  meet  training  objectives 
than  with  a  didactic  curriculum  alone,  but  provide  a  hands-on  alternative  to  moulage  field 
exercises  in  a  less  expensive  and  more  readily  available  platform.  Both  manikin-based  and 
virtual  reality  training  curriculums  can  be  integrated  into  a  multidisciplinary  training  program  to 
enhance  the  training  of  first  responders. 

Through  this  proposal,  we  have  conducted  the  following  studies: 

•  Heuristic  Usability  Evaluations  -  The  software  development  and  evaluation  model  that  we 
have  developed  consists  of  two  iterations  of  expert  heuristic  testing  and  one  evaluation  using 
typical  users  of  the  system.  As  key  to  the  success  of  VR  Nephron,  it  is  vitally  important  that 
the  application  is  usable  and  able  to  satisfy  the  needs  of  the  training,  without  hindrance  of  the 
technology  itself.  Therefore,  in  the  development  process  of  VR  Nephron,  we  have  developed 
and  utilized  a  heuristic  usability  evaluation  model. 

Conclusion  -  We  believe  that  simulations  will  provide  an  increasingly  realistic  and  effective 
method  of  teaching  concepts  to  medical/healthcare  students.  Human  physiology  is  very 
complex,  and  still  not  completely  understood.  Creating  models  and  performing  simulations 
has  the  potential  to  help  us  gain  a  more  complete  understanding  of  how  the  body  works,  and 
ultimately  improve  patient  care. 

The  modeling  of  the  physiologic  variables,  as  well  as  the  addition  of  stylistic  components  of 
gaming  motifs,  remains  a  work  in  progress.  Based  on  feedback  from  the  students,  the  current 
version  of  the  nephron  game  is  probably  most  suitable  for  novice  learners  of  renal 
physiology.  Although  users  felt  that  the  interface  was  easy  to  learn,  the  present  software  is 
probably  optimally  used  with  a  knowledgeable  technical  facilitator  present.  The 
development  of  the  virtual  reality  application  for  use  in  front  of  a  large  3D  screen  would 
make  the  system  amenable  to  greater  integration  into  a  problem-based  learning  curriculum 
that  emphasizes  small  group  interaction  and  formative  assessment  facilitated  by  a  faculty 
member. 
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Future  studies  would  be  valuable  to  assess  the  contribution  of  gaming  elements  to  student 
learning  and  satisfaction,  as  well  as  to  compare  the  impact  of  immersion  in  this  virtual 
system  to  traditional  non-immersive  forms  of  education. 

•  Manikin  Triage  Training  -This  research  study  that  will  help  us  understand  the  optimal 
conduct  of  triage  training  using  manikin  simulators,  and  that  packaging  and  disseminating 
sophisticated  manikin-based  online  curricula  can  used  as  a  more  readily  available  and  less 
expensive  alternative  to  mass  casualty  triage  training. 

Conclusion  -  Novice  learners  demonstrated  improved  Triage  and  Intervention  Scores,  speed, 
and  self-efficacy  during  an  iterative,  fully  immersed  VR  triage  experience. 

•  VRMSS  -  The  VRMSS  system  has  the  potential  to  facilitate  an  inexpensive  alternative  to  the 
current  traditional  methods  of  training,  and  provides  a  means  to  train  laparoscopic  motor 
skills  which  will  improve  patient  safety  for  military  personnel.  Expensive  desktop  models  are 
valuable,  but  due  to  their  high  cost,  the  value  of  simulation  training  is  experienced  by 
relatively  few  surgeons.  Once  this  baseline  application  is  validated  locally,  a  distributed 
server-based  model  using  inexpensive  tools  would  help  to  improve  access  to  this  type  of 
training. 

Conclusion  -  With  the  VRMSS  and  box  trainer  groups  having  similar  results  on  the  LapSim 
assessment,  it  is  likely  that  VRMSS  would  also  have  a  positive  impact  on  outcomes  at 
surgery.  However,  like  LapSim,  VRMSS  has  significant  advantages  over  the  box  trainer,  in 
that  the  VRMSS  can  provide  scoring  on  several  parameters  without  the  need  of  an  instructor 
during  student  skill  acquisition.  Parameters  of  importance  scored  by  the  VRMSS  include: 
time  to  complete  task,  number  of  errors  and  economy  of  movements.  Also,  the  VRMSS  is 
approximately  1/1 6th  the  cost  of  the  LapSim. 

The  SimCenter  TELIS  model  is  designed  to  ultimately  facilitate  a  smooth  development  and 
integration  of  simulation  technology  into  education  and  training.  Our  design  model  provides  an 
accepted  methodology  for  development  and  validation  of  innovative  technological  learning 
systems.  The  goal  of  the  model  is  to  produce  validated  TELIS  to  enhance  learning  with  all  the 
encumbrances  worked  out  through  user  testing  and  validation  studies.  In  addition  to  the 
systematic  approach  to  development,  SimCenter  Hawaii  is  proposing  a  broad  portfolio  of 
research  that  can  contribute  to  greater  educational  outcomes  in  healthcare. 

SimCenter  Hawaii  is  motivated  by  the  Research  &  Development  Roadmap  published  by  the 
Learning  Federation, (3)  which  is  a  partnership  between  industry,  academia  and  private 
foundations.  In  essence,  SimCenter  Hawaii’s  aim  is  to  promote  and  identify  research  and 
development  that  can  directly  lead  to  improved  learning  outcomes  and  greater  access  to  quality 
education  and  training  in  healthcare  while  promoting  collaboration  between  different  entities. 

Ultimately,  SimCenter  Hawaii’s  goals  for  the  development  of  TELIS  are  not  only  to  foster  a 
proven  research  and  design  methodology,  but  to  also  cultivate  innovative  and  collaborative 
research  and  development,  and  to  seed  further  growth  and  development  in  the  Hawaii  Pacific 
region.  We  hope  not  only  to  play  a  pivotal  role  in  research  and  development  methodology,  but 
also  to  generate  partnerships  with  private  companies,  universities  government  agencies,  private 
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foundations  and  the  educational  community.  Through  our  research  and  design  model  for  TELIS, 
we  can  provide  a  platform  for  scientists,  clinicians,  engineers,  industry  and  government  to 
nurture  and  develop  innovative  research  and  solutions  in  healthcare. 


Page  19  of  86 


W81XWH-07-2-0062  12/12/2008 

SimCenter  Hawaii:  Virtual  Reality  Applications  for  Health  Care  Education  and  Training 


References 


1 .  Component  Roadmap:  Building  Simulations  &  Exploration  Environments  R&D 
Technology-Enabled  Learning  Systems.  2003.  Washington,  DC. 

Ref  Type:  Report 

2.  Component  Roadmap:  Instructional  Design  in  Technology-Enabled  Learning  Systems: 
Using  Simulations  and  Games  in  Learning.  2003.  Washington,  DC,  The  Learning 
Federation  Project. 

Ref  Type:  Report 

3.  The  Learning  Federation:  Learning  Science  and  Technology  Research  &  Development 
Roadmap,  Executive  Summary.  2003.  Washington,  DC,  The  Learning  Federation  Project. 
Ref  Type:  Report 

4.  Alverson  DC,  Saiki  SM,  Jr.,  Jacobs  J,  Saland  L,  Keep  MF,  Norenberg  J,  Baker  R,  Nakatsu 
C,  Kalishman  S,  Lindberg  M,  Wax  D,  Mowafi  M,  Summers  KL,  Holten  JR,  Greenfield  JA, 
Aalseth  E,  Nickles  D,  Sherstyuk  A,  Haines  K,  Caudell  TP.  Distributed  interactive  virtual 
environments  for  collaborative  experiential  learning  and  training  independent  of  distance 
over  Intemet2.  Stud  Health  Technol  Inform  98:  7-12,  2004. 

5.  Aschwanden  C,  Sherstyuk  A,  Burgess  L,  Montgomery  K.  A  Surgical  and  Fine-Motor  Skills 
Trainer  for  Everyone?  Touch  and  Force-Feedback  in  a  Virtual  Reality  Environment  for 
Surgical  Training.  Medicine  Meets  Virtual  Reality  2006.  Medicine  Meets  Virtual  Reality 
14 , 19-21.  2006.  IOS  Press.  1-24-2006. 

Ref  Type:  Conference  Proceeding 

6.  Dawson-Saunders  B,  Feltovich  PJ,  Coulson  RL,  Steward  DE.  A  survey  of  medical  school 
teachers  to  identify  basic  biomedical  concepts  medical  students  should  understand. 
Academic  Medicine:  Journal  Of  The  Association  Of  American  Medical  Colleges  65:  448- 
54,  1990. 

7.  Garfield  JM,  Paskin  S,  Philip  JH.  An  evaluation  of  the  effectiveness  of  a  computer 
simulation  of  anaesthetic  uptake  and  distribution  as  a  teaching  tool.  Med  Educ  23:  457-62, 
1989. 

8.  Issenberg  SB,  Gordon  MS,  Gordon  DL,  Safford  RE,  Hart  IR.  Simulation  and  new  learning 
technologies.  Med  Teach  23:  16-23,  2001. 

9.  Jordan  JA,  Gallagher  AG,  McGuigan  J,  McClure  N.  Virtual  reality  training  leads  to  faster 
adaptation  to  the  novel  psychomotor  restrictions  encountered  by  laparoscopic  surgeons. 
Surg  Endosc  15:  1080-4,  2001. 

10.  Kelly  H,  Blackwood  V,  Roper  M,  Higgins  G,  Klein  G,  Tyler  J,  Fletcher  D,  Jenkins  H, 

Chisolm  A,  Squire  K.  Training  Technology  against  Terror:  Using  Advanced  Technolgy  to 
Prepare  America's  Emergency  Medical  Personnel  and  First  Responders  for  a  Weapon  of 


Page  20  of  86 


W81XWH-07-2-0062  12/12/2008 

SimCenter  Hawaii:  Virtual  Reality  Applications  for  Health  Care  Education  and  Training 


Mass  Destruction  Attack.  1-44.  2002.  Washinton,  DC,  The  Learning  Federation. 

Ref  Type:  Report 

11.  Langelotz  C,  Kilian  M,  Paul  C,  Schwenk  W.  LapSim  virtual  reality  laparoscopic  simulator 
reflects  clinical  experience  in  German  surgeons.  Langenbecks  Arch  Surg  390:  534-7,  2005. 

12.  Langelotz  C,  Kilian  M,  Paul  C,  Schwenk  W.  LapSim  virtual  reality  laparoscopic  simulator 
reflects  clinical  experience  in  German  surgeons.  Langenbecks  Arch  Surg  390:  534-7,  2005. 

13.  Melvin  WS,  Johnson  JA,  Ellison  EC.  Laparoscopic  skills  enhancement.  Am  J  Surg  172: 
377-9,  1996. 

14.  Nielsen  J.  Usability  Engineering.  San  Diego:  Academic  Press,  1993. 

15.  Sullivan  B.  Emergency  training  systems  -  a  survey.  2006.  Washington  DC,  The  Learning 
Federation  Project,  Federation  of  American  Scientists. 

Ref  Type:  Report 


Page  21  of  86 


W81XWH-07-2-0062  12/12/2008 

SimCenter  Hawaii:  Virtual  Reality  Applications  for  Health  Care  Education  and  Training 


Appendices 

Appendix  A.  Publication  -  Wang  K,  Reed  N,  Vincent  D.  (2009).  Multi-Agent  Simulation  of 
Kidney  Function.  In  Multi- Agent  Systems  for  Health  Care  Simulation  and 
Modeling.  IGI  Global. 

Appendix  B.  Publication  -  Vincent  D,  Sherstyuk  A,  Burgess,  Connolly  K.  (2008).  Teaching 
Mass  Casualty  Triage  Using  Immersive  Three-dimensional  Virtual  Reality.  Acad 
Emer  Med  2008;  15:1-6 

Appendix  C.  Publication  -  Ikehara  C,  Aschwanden  C,  Burgess  L,  Montgomery  K,  Mok  D. 

(2009).  Evaluating  a  Virtual  Reality  Motor-Skills  Simulator.  MMVR  2009 

Appendix  D.  Presentation  -  Overview  of  Projects 


Page  22  of  86 


Appendix  A 


A  Multi-Agent  Simulation  of  Kidney 
Function  for  Medical  Education 


Kin  Lik  Wang 

Telehealth  Research  Institute ,  University  of  Hawaii ,  USA 

Nancy  E.  Reed 

Department  of  Information  and  Computer  Sciences,  University  of  Hawaii,  USA 

Dale  S.  Vincent 

Telehealth  Research  Institute,  University  of  Hawaii,  USA 


ABSTRACT 

This  chapter  describes  a  multi-agent  system  to  simulate  kidney  function  for  the  purpose  of  teaching  renal 
physiology  to  healthcare  students.  Renal  function  is  modeled  with  particles  that  move  within  an 
environment,  using  the  agent  paradigm.  Particles  represent  molecules  and  fluids  and  the  environment 
represents  the  structures,  membranes  and  volumes  of  the  kidneys.  Particles  move  dynamically  through  the 
system,  responding  appropriately  depending  on  their  surroundings.  We  describe  the  use  of  the  simulation 
in  research  and  teaching  medical  students  about  the  renal  system.  Initial  results  of  heuristic  and  usability 
testing  by  medical  students  demonstrated  that  the  3D  virtual  reality  educational  tool  developed  improves 
the  visualization  of  the  renal  system  by  medical  students,  and  student  self-confidence  in  learning  renal 
physiology. 
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INTRODUCTION 

Renal  physiology  is  exceptionally  difficult  to  learn.  The  countercurrent  exchange  system  of  the  kidney 
was  ranked  number  one  in  basic  science  difficulty  and  number  two  in  clinical  difficulty  in  a  1990  survey 
of  medical  educators  (Dawson-Saunders,  1990).  Project  TOUCH  was  a  multiyear  collaboration  between 
the  University  of  Hawaii  and  the  University  of  New  Mexico  in  which  virtual  reality  (VR)  applications 
were  developed  to  advance  medical  education.  One  application  was  a  3D  VR  fly-through  model  of  the 
kidney  (Alverson  &  Saiki,  2006).  The  simulation  prototype  has  subsequently  been  enhanced  with  a  new 
user  interface  and  gaming  motifs.  The  new  system  underwent  heuristic  and  usability  tests  that  are 
described  in  this  chapter. 

The  kidney's  function  is  to  eliminate  water  soluble  wastes  from  the  body  (Banasik,  2000).  We  chose  a 
particle  system  implementation  because  that  produces  a  better,  more  detailed  simulation.  A  multi-agent 
system  represents  the  motion  of  molecules  and  the  functions  of  the  tubules  in  a  kidney.  The  particles 
follow  the  tubule  path  dynamically  using  intelligent  movements  to  simulate  different  kidney  function, 
such  as  reabsorption  and  secretion. 
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The  rest  of  this  paper  is  organized  as  follows.  First,  multiagent  systems  and  the  renal  system  are 
described.  Second,  we  describe  the  implementation  and  characteristics  of  our  simulation  system.  Then 
the  visualization  of  difficult  concepts  to  aid  understanding  is  describes.  Finally,  the  usability  experiments 
conducted  and  the  results  obtained  are  described  followed  by  future  trends  for  agents  in  medical 
simulations  and  the  conclusion. 

BACKGROUND 
Multi-Agent  Systems 

A  multi-agent  system  (MAS)  is  composed  of  agents,  each  of  which  selects  its  own  course  of  action  based 
upon  its  goals  (Weiss,  1999,  Wooldridge,  2002).  Most  definitions  of  agents  include  the  following 
characteristics:  a)  An  agent  perceives  its  environment,  decides  on  actions,  and  uses  its  effectors  to 
perform  the  selected  actions,  b)  An  agent  is  active  over  a  period  of  time,  making  actions  based  on  the 
world  it  senses  at  the  current  time,  c)  An  agent  has  goal(s)  as  well  as  some  latitude  as  to  how  to  achieve 
those  goals  (Barber  et  al.  2003).  A  multi-agent  system  may  be  incorporated  in  a  physical  (robotic)  body, 
or  sense  and  act  upon  the  world  entirely  in  software  simulations,  or  computer  applications  (softbots). 

Multiagent  systems  have  been  used  in  a  broad  range  of  applications  including  soccer  games  (Robocup, 
2008),  search  and  rescue  operations  (Robocup  Rescue,  2008),  space  (Bernard  et  al.,  1999,  NASA  2008), 
and  military  applications  (Zafar,  2006),  including  training  combat  pilots,  troops  in  battlefield  scenarios, 
and  numerous  training  situations.  Agents  have  been  used  in  medical  applications  as  well.  The  journal 
Artificial  Intelligence  in  Medicine  recently  published  a  special  issue  on  software  agents  in  health  care 
(Moreno  &  Garbay,  2005).  Experience  with  agent  systems  in  the  GruSMA  research  group  is  described  by 
Moreno,  Vails,  Isem,  &  Sanchez  (2006). 

Examples  of  agents  in  medicine  include  Hudson  &  Cohen  (2006),  who  use  agents  to  support  rural 
healthcare  of  the  elderly.  Poulopoulos,  et  al.  (2001)  use  intelligent  agents  to  track  patient  information  in 
large  databases  for  use  in  a  telemedicine  system.  Prado  et  al.  (2001)  describe  a  telemedicine  system  to 
support  patients  with  end  stage  renal  disease.  This  system  focuses  on  the  monitoring  and 
treatment/therapy  for  patients  with  advanced  renal  disease. 

Most  agent  applications  in  medicine  are  focused  on  decision-support  for  physicians  and/or  patients.  In 
contrast,  the  goal  of  our  system  is  to  teach  physiological  concepts  to  health  care  professionals,.  When 
health  care  workers  have  a  better  understanding  of  the  physiology  of  the  human  body,  they  can  also 
understand  the  effects  of  disease  processes  in  patients,  which  can  improve  the  quality  of  care  delivered  to 
the  patient.  Our  program  encourages  the  user  to  experiment  by  changing  parameters  and  viewing  the 
results  in  simulation. 

Why  use  agents  for  this  project?  There  are  many  ways  to  represent  moving  molecules,  which  form  the 
base  for  a  renal  simulation.  Reeves  (1983)  introduced  a  particle  system  method  for  modeling  fuzzy 
objects  such  as  fire,  clouds,  and  water.  The  resulting  model  is  able  to  represent  motion,  changes  of  form, 
and  dynamics,  but  this  approach  does  not  produce  the  randomness  needed  in  the  kidney  domain. 

Chenney  (2004)  used  flow  tiles  for  representing  and  designing  velocity  fields.  Each  flow  tile  contains 
small  fields,  and  many  tiles  can  be  combined  to  produce  a  larger  flow.  He  describes  three  applications:  a 
crowd  on  a  city  street,  a  river  flowing  between  banks,  and  swirling  fog.  We  can  apply  the  flow  tile 
concept  to  the  particle,  but  the  problem  is  that  flow  tiles  use  a  pre-defined  path,  so  it  is  hard  to  use  flow 
tiles  to  do  reabsorption  and  secretion  that  are  necessary  in  the  kidney  system. 

One  type  of  multiagent  system  algorithm  is  swarm  intelligence.  Baumgartner  (2004)  proposed  particle 
swarm  optimization  which  imitates  the  social  behavior  of  birds  in  a  flock  flying  around  and  sitting  down 
on  a  pylon.  Yoon  (2005)  describes  a  tool  to  generate  way  finding  aids  in  dynamic  virtual  worlds  in  an  ant 
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colony.  Ants  look  for  food  by  wandering  randomly  from  their  home  colony.  If  they  find  food,  they  return 
to  their  colony  while  laying  down  pheromone  (scent)  trails  that  can  be  followed  by  other  ants.  The 
pheromones  are  a  simple  but  effective  means  of  communication.  Ants  are  behaviorally  unsophisticated 
and  yet,  collectively,  they  perform  complex  tasks.  The  ant  colony  algorithm  can  be  run  continuously  and 
adapt  to  changes  in  real  time.  In  another  example,  Schmickl  (2005)  modeled  honey  bees  to  forage  for 
nectar  in  a  fluctuating  environment.  Without  any  global  decision  making,  the  bees  are  able  to  select 
optimal  paths,  those  offering  the  best  ratio  of  gain  to  cost  to  and  from  multiple  available  nectar  sources. 

The  swarm  intelligence  models  have  an  advantage  over  simulated  annealing  and  genetic  algorithm 
approaches  when  the  environment  may  change  dynamically  (Janson,  2008).  They  are  computationally 
expensive,  which  could  prevent  the  simulation  from  running  in  “real  time”. 

Using  the  ant  colony  analogy,  the  particles  in  our  system  are  the  ants  (agents)  with  simple  behavior 
rules.  Their  behavior  produces  the  movement  of  the  particles  and  fluids  in  our  simulation.  Even  though 
the  actions  of  the  particles  individually  are  simple,  the  result  is  a  complex  system.  The  agent  paradigm 
provides  a  distributed  framework  for  robustness  and  efficiency.  The  graphics  engine  displays  the 
environment  and  the  movement  of  .the  particles  in  3D. 


The  structure  and  function  of  the  kidneys 

Humans  have  two  kidneys  that  each  contains  more  than  a  million  nephrons.  A  nephron  contains  eight 
functional  parts  as  shown  in  Figure  1:  (A)  glomerulus,  (B)  Bowman’s  capsule,  (C)  proximal  convoluted 
tubule,  (D)  descending  loop  of  Henle,  (E)  ascending  loop  of  Henle,  (F)  distal  convoluted  tubule,  (G)  the 
collecting  tubule,  and  (H)  vasa  recta.  Thus,  fluid  flows  from  A  -  G  in  the  figure.  The  parts  and  functions 
of  the  kidneys  are  described  next. 

Blood  from  the  body  enters  nephrons  in  the  kidney  from  the  renal  artery.  Urea  is  a  waste  product  of 
metabolism  and  is  carried  by  the  blood  to  the  kidneys.  The  movement  of  urea  is  responsive  to  the 
hormone  aldosterone.  The  kidneys  are  responsible  for  removing  waste  products  from  the  blood,  however 
when  removing  the  wastes,  electrolytes  and  water  are  also  removed,  and  thus  must  be  recovered.  The 
process  occurs  in  the  following  five  steps. 

The  concentration  of  urine  permits  animals  to  survive  on  much  less  water  than  they  would  otherwise 
need  to  get  rid  of  metabolic  waste  products.  This  process  is  unique  to  mammals. 
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Figure  1:  The  eight  parts  of  a  nephron  in  a  kidney.  (A)  glomerulus,  (B)  Bowman’s  capsule,  (C)  proximal 
convoluted  tubule,  (D)  descending  loop  of  Henle,  (E)  ascending  loop  of  Henle,  (F)  distal  convoluted 
tubule,  (G)  the  collecting  tubule,  and  (H)  vasa  recta  (blood  vessels). 

First,  the  glomerulus  (A  in  Figure  1)  filters  fluid  (plasma)  from  the  blood  into  Bowman’s  capsule  (B) 
while  preventing  the  passage  of  blood  cells  and  proteins  from  the  blood.  Second,  the  proximal 
convoluted  tubule  (C)  reabsorbs  two  thirds  of  the  filtered  water  and  electrolytes  and  all  of  the  filtered 
bicarbonate,  glucose,  amino  acids,  and  vitamins.  Third,  the  descending  loop  of  Henle  (D)  reabsorbs  more 
water  and  delivers  a  concentrated  filtrate  to  the  ascending  loop  of  Henle  (E).  Fourth,  the  ascending  loop 
of  Henle  actively  reabsorbs  sodium  ions  to  produce  a  concentrated  filtrate.  Fifth,  the  distal  convoluted 
tubule  (F)  reabsorbs  additional  sodium  ions,  chlorine  ions,  and  water.  Finally,  the  collecting  tubule  (G) 
reabsorbs  water  under  the  influence  of  antidiuretic  hormone,  and  secretes  hydrogen  ions  and  potassium 
ions. 

Thomas  et  al.  (2006)  provide  a  summary  of  the  contributions  of  mathematical  models  of  the  kidney. 
The  models  are  at  various  levels  of  detail  and  their  purpose  is  to  obtain  a  comprehensive  understanding  of 
renal  function.  To  that  end,  the  authors  are  creating  new  resources  (given  the  name  Physiome)  to  provide 
general  access  to  current  and  future  models  to  enhance  collaboration  between  researchers. 
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DESIGN  AND  IMPLEMENTATION  OF  THE  MULTI-AGENT  SYSTEM 


This  research  was  initially  funded  by  Telehealth  Outreach  for  Unified  Community  Health  (TOUCH) 
(Alverson  &  Saiki,  2008)  and  implemented  in  the  software  development  environment  called  Flatland 
(Flatland,  2008).  The  aim  of  Project  TOUCH  was  to  help  develop  a  virtual  reality  simulation  for  medical 
education  (Alverson  &  Saiki,  2005). 

This  chapter  describes  further  work  on  the  system  including  the  addition  of  gaming  motifs,  as  well  as 
heuristic  and  usability  testing.  This  work  was  supported  under  a  grant  to  the  University  of  Hawaii 
Telehealth  Research  Institute  by  the  U.S.  Army’s  Telemedicine  and  Advanced  Technology  Research 
Center  (TATRC)  to  develop  virtual  reality  applications. 


Figure  2:  A  screenshot  of  our  simulation ,  showing  the  structure  of  nephrons  and  molecules  in  the  kidney. 
The  small  dots  show  the  location  of  particles  within  the  kidney. 

Figure  2  shows  user’s  simulation  view  of  three  nephrons.  Each  nephron  is  a  multi-agent  system 
consisting  of  an  environment  and  agents  (e.g.  particles).  Kidney  processes  are  displayed  when  the 
molecules  move  within  the  environment.  This  uses  one  of  the  plugins  in  Flatland  (Flatland,  2008; 
Alverson&  Saiki,  2006).  The  system  is  written  in  the  C  programming  language  with  OpenGL. 

Each  particle  in  the  system  is  an  agent.  The  agent's  sensors  obtain  the  particle's  current  location  in  its 
environment.  Each  agent/particle  decides  the  direction  and  velocity  it  will  move  (if  any)  during  the  step 
in  the  simulation.  Particles  thus  move  through  the  environment  modeling  the  physiological  processes. 

Our  system  is  analogous  to  trains  within  a  railroad  system.  The  train  system  has  stations  and  tracks  in 
its  environment,  cars  and  passengers  can  be  in  trains  or  in  stations,  and  passenger  traffic  represents  the 
flow  of  molecules  within  the  system. 
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The  Environment:  Nodes  and  Edges 

A  node  is  analogous  to  a  train  station,  while  an  edge  is  analogous  to  tracks.  Agents  move  from  node  to 
node  along  the  edges.  The  agents  can  only  move  where  an  edge  exists.  Each  node  has  a  3D  plane 
perpendicular  to  the  edge  attached  to  the  previous  node.  The  dot  product  of  the  particle  coordinate  and 
the  3D  plane  normal  vector  determine  whether  a  particle  passes  through  the  node  or  not. 


Agents:  Particles  and  Drams 

There  are  two  types  of  agents:  particles  and  drams.  Particles  model  either  molecules  including  water, 
sodium,  urea,  and  potassium,  or  fluid,  including  blood,  plasma,  water  and  filtrates.  A  particle  has  the 
following  attributes:  an  identifier,  type,  location,  velocity  and  direction.  We  group  particles  into  drams  to 
simplify  the  simulation  since  groups  of  agents  of  multiple  types  move  through  the  system  together. 
Particles  can  enter  or  exit  a  dram  or  other  structure  during  the  simulation. 

A  dram  is  a  group  of  particles  and/or  fluids,  representing  a  volume  of  liquid.  Drams  have  attributes 
including  an  identifier,  their  current  location  within  the  environment,  their  current  direction  of  motion, 
and  their  current  speed.  Drams  move  through  the  environment  over  time.  Their  motion  represents  the 
flow  of  liquid  through  the  renal  system.  The  ratio  of  reabsorption  and  secretion  of  molecules  and  fluids  is 
based  on  characteristics  of  the  molecule  and  environment.  By  using  the  dot  product,  a  group  of  particles 
or  drams  can  act  like  an  agent  to  perceive  the  node  and  edge  for  actions  such  as  reabsorption  or  secretion. 

The  particle  pool  is  used  for  showing  the  reabsorption  and  secretion  functions  in  the  kidney.  There  are 
two  kinds  of  particle  pools:  a  temporary  pool  to  simulate  reabsorption,  and  a  stay  pool  to  simulate 
secretion.  The  temporary  pool  is  used  when  the  particle  moves  from  inside  the  tubule  (lumen)  to  outside 
the  tubule  (inter stitium);  the  particle  disappears  to  show  movement  into  the  interstitium  for  reabsorption. 
The  stay  pool  uses  a  preexisting  particle  from  outside  the  tubule  (interstitium)  to  move  to  inside  the  tubule 
(lumen);  the  particle  then  joins  the  flow  inside  the  tubule  to  simulate  the  secretion. 


Processes:  Flow 

The  flow  of  agents  in  the  system  is  produced  by  six  operations:  enter,  update,  separate,  reabsorb ,  secrete , 
and  transfer.  A  group  of  particles  can  enter  to  a  dram  and  form  a  flow  inside  the  tubule.  As  particles 
move  through  the  tubule,  they  can  move  in  different  directions;  separate  a  dram  into  two  different  drams; 
reabsorb  from  a  dram  to  a  pool;  secrete  from  a  pool  into  a  dram;  or  transfer  to  other  parts  of  the  nephron. 
The  particles  and  drams  update  their  current  positions  at  each  step  in  the  simulation.  The  transfer 
operation  is  used  to  show  the  urea  gradient  in  the  loop  of  Henle.  The  urea  in  the  collecting  tubule  can 
flow  back  to  the  loop  of  Henle  by  moving  the  urea  particle  to  a  gradient  pool.  The  gradient  pool  is  a  stay 
pool  that  demonstrates  how  a  urea  gradient  can  form  in  the  kidney. 


KIDNEY  FUNCTIONS 
Filtration 

The  filtration  process  occurs  when  small,  usually  uncharged,  molecules  move  from  the  blood  in 
glomerular  capillaries  to  the  Bowman's  capsule.  Other  blood  components  such  as  red  or  white  blood  cells 
remain  in  the  blood  and  exit  the  nephron  without  going  through  the  tubules.  The  plasma  and  particles 
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recovered  in  the  tubules  are  added  to  blood  that  bypasses  the  filtration  process.  The  filtrate  contains 
WUSP,  which  stands  for  molecules  of  water,  urea,  sodium,  and  potassium  (blood  plasma). 

There  are  two  paths  out  of  the  glomerulus  (A),  one  path  to  the  Bowman's  capsule  (B),  and  the  other  to 
the  vasa  recta  (H).  In  the  glomerulus  node,  agents  sense  the  information  about  their  environment,  and 
choose  one  of  the  available  paths.  Then,  the  dram  is  split  into  two  drams  with  filtered  blood  in  one  dram 
and  WUSP  in  the  other  dram.  In  this  way,  blood  goes  directly  to  the  vasa  recta  (H)  where  it  returns  to  the 
body  and  is  combined  with  recovered  particles  (WUSP). 

Reabsorption 

The  WUSP  particles  actively  or  passively  leave  the  tubule,  such  as  proximal  tubule  (C),  descending  loop 
of  Henle  (D),  and  ascending  loop  of  Henle  (E),  reenter  the  plasma  in  the  process  called  reabsorption.  In 
the  reabsorption  node,  a  dram  leaves  the  tubule  its  current  status  is  obtained  and  it  is  absorbed 
(disappears).  Next  the  particles  move  from  the  dram  into  a  particle  pool,  and  are  reabsorbed. 

Secretion 

The  potassium  particles  move  from  the  plasma  into  the  collecting  tubule  (G),  primarily  by  active 
transport.  In  active  transport,  energy  is  expended  in  the  cells  to  force  molecules  through  a  barrier.  Passive 
transport  (diffusion)  also  occurs.  This  maintains  homeostasis  (pH  and  other  properties  of  the  blood  to 
normal  values)  and  rid  the  body  of  metabolic  by-products  and  toxins.  In  the  simulation,  pre-defined 
particle  pools  are  available.  When  needed,  the  particles  move  into  the  tubule. 

Excretion 

The  particles  remaining  in  the  distal  tubule  (F)  after  passing  through  the  collecting  tubule  (G)  leave  the 
body  as  urine,  the  process  called  excretion.  Excretion  is  modeled  by  particles  going  through  the 
collecting  tubule  to  the  bladder.  In  the  bladder,  the  dram  dissolves  in  the  flow  and  all  particles  inside  are 
deleted. 

Countercurrent  mechanism 

A  countercurrent  exchange  occurs  when  fluid  currents  flow  opposite  of  each  other  and  a  property  (such  as 
concentration  of  a  substance)  is  exchanged.  There  are  several  countercurrent  flows  demonstrated  in  the 
simulation.  The  descending  loop  of  Henle  (D)  and  ascending  loop  of  Henle  (E)  are  adjacent  structures, 
and  particles  flow  in  opposite  directions  and  are  exchanged.  Other  countercurrent  exchanges  occur 
between  the  ascending  loop  of  Henle  (E)  and  collecting  tubule  (G)  in  the  simulation.  Adjacent  structures 
that  also  have  countercurrent  flows,  such  as  the  descending  loop  of  Henle  (D)  and  vasa  recta  (H),  and 
ascending  loop  of  Henle  (E)  and  vasa  recta  (H),  are  depicted  in  a  static  fashion  in  the  model.  The 
movement  of  particles  in  the  simulation  gives  users  a  dynamic  way  to  visualize  this  difficult  physiologic 
concept. 


VISUALIZATION  SYSTEM  TOOLS 
Menu 

The  user’s  menu  is  shown  in  Figure  3.  Parameters  are  easy  to  control  using  a  paddle,  and  changes  can  be 
made  during  the  simulation.  The  paddle  interface  is  a  fan  with  an  attached  inertia  cube  (InterSense,  2008) 
to  detect  the  rotation  and  move  the  laser  pointer  in  order  to  select  the  items  as  shown  as  a  small  circular 
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dot  in  Figures  2  and  3.  The  student  is  holding  the  paddle  in  Figure  4.  In  the  menu,  the  user  has  the  ability 
to  select  which  particles  to  show  in  the  simulation  (such  as  water,  urea,  sodium,  potassium,  or/and  blood). 
The  user  can  also  select  whether  to  start  or  stop  the  flow  of  particles.  The  gaming  mode  allows  the  user  to 
explore,  question,  walkthrough,  or  jump  to  different  sequence.  The  explore  mode  can  highlight  and 
display  the  different  parts  of  nephron  game.  The  question  mode  presents  a  question  to  a  user,  and  allows 
the  user  to  select  the  correct  part  of  the  nephron  with  the  paddle.  The  walkthrough  mode  runs  a  pre¬ 
defined  path,  enabling  the  user  to  view  the  different  parts  of  nephron  from  a  different  camera  angle.  In 
the  sequence  mode,  a  user  is  able  to  jump  to  15  different  locations  directly,  and  switch  between  a 
mechanical  view  and  an  organic  view  of  the  nephron.  The  mechanical  view  can  give  medical  students  a 
visual  representation  of  the  function  of  the  nephron.  The  organic  view  shows  a  biological  representation 
of  the  nephron. 


Figure  3:  The  menu  enables  the  user  to  choose  which  particles  to  display,  as  well  as  control  other 
parameters  during  a  simulation. 


VISUALIZATION  OF  DIFFICULT  CONCEPTS 
Filtration  in  Bowman’s  capsule 

Small  particles  are  filtered  from  the  blood  into  a  tubule  in  the  kidneys  via  a  semi-permeable 
membrane  in  Bowman’s  capsule  (B  in  Figure  1).  This  is  visualized  by  showing  the  particles  of 
Urea,  Sodium  and  Potassium  along  with  water  moving  out  of  the  blood  coming  into  the  kidneys 
and  into  a  tubule.  Most  of  the  blood,  containing  large  cells  and  proteins,  continues  to  flow 
through  the  small  vessels  in  the  nephron  and  eventually  leaves  the  kidneys. 

Initially  the  size  of  the  particles  was  proportional  to  their  actual  size.  Urea  is  the  largest  and 
sodium  is  the  smallest.  Users  mentioned  that  they  could  not  see  the  Na+  particles,  so  their  size 
was  increased. 
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Reabsorption  of  water  and  electrolytes 

The  reabsorption  of  water  and  electrolytes  is  visualized  when  the  particles  rejoin  the  blood  after 
passing  through  the  kidneys.  The  particles  “disappear”  into  the  blood. 

Counter-current  exchange  and  multiplication 

The  structures  in  the  tubules  generate  a  gradient  where  the  concentration  of  particles  increases 
(increasing  osmolality).  This  occurs  in  the  loop  of  Henle,  in  an  elegant  manner  using  the 
opposite  direction  fluid  flows  in  the  adjacent  parts  of  the  tubule  to  concentrate  the  urine.  It  is 
somewhat  like  a  heat  exchanger,  which  looks  much  like  an  automobile  radiator,  on  an  air  vent  in 
a  building.  The  intake  and  outflow  tubes  are  next  to  each  other.  In  cold  climates,  the  incoming 
cold  air  is  warmed  by  the  warm  air  going  out.  At  the  same  time,  the  air  that  was  warm  in  the 
building  is  cooled  before  it  exits.  Thus,  heat  is  retained  in  the  building  rather  than  expelled  with 
the  air  when  bringing  fresh  air  into  the  building.  Without  adjacent  inflow  and  outflow,  the  heat 
would  be  lost  with  the  air  as  it  is  expelled. 

This  is  one  of  the  key  concepts  to  understand  about  the  renal  system.  Our  simulation  does  not 
visually  represent  the  gradients  in  the  kidneys.  We  would  like  to  include  this  in  future  versions 
of  the  system.  A  change  in  color  intensity  could  represent  different  levels  of  particles  at  the 
different  parts  of  the  gradient. 


USABILITY  STUDIES 
Study  Design 

The  studies  took  place  at  a  medical  school  simulation  center.  Data  for  the  group  user  evaluation  were 
collected  anonymously  using  an  audience  response  polling  system.  Data  for  the  heuristic  evaluation  and 
the  individual  user  evaluation  were  collected  through  interviews  and  written  surveys.  The  research 
protocols  were  approved  by  the  University  of  Hawaii  Committee  on  Human  Studies  and  the  United  States 
Army  Medical  Materiel  and  Research  Command  Office  of  Research  Protection. 

Group  User  Evaluation 

First  year  medical  students  viewed  the  virtual  reality  nephron  during  planned  laboratory  sessions  that 
were  a  part  of  their  problem-based  learning  curriculum.  All  participants  donned  active  3D  glasses,  and  a 
single  student  used  a  paddle  interface  to  navigate  the  scene  as  it  was  projected  on  a  rear  mounted  3D 
projector.  One  or  two  faculty  members  mediated  each  45-minute  session.  Students  were  surveyed  before 
and  after  the  session  using  an  audience  response  system  that  collected  anonymous,  paired  responses. 
Responses  were  rated  using  a  5 -point  Likert  scale,  from  1  =  strongly  disagree  to  5  =  strongly  agree  to  the 
following  statements: 

1)  “I  feel  confident  in  my  understanding  of  nephron  physiology”; 

2)  I  have  a  good  mental  image  of  renal  physiology”;  and 

3)  “Renal  physiology  is  hard  to  visualize.” 
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Observations  and  comments  made  during  the  group  session  were  collated  and  organized  into  action  items 
for  the  software  programmers.  The  programmers  subsequently  added  gaming  motifs,  on-screen 
navigation  controls,  a  simple  scoring  system,  as  well  as  enhancements  in  the  visualization  of  selected 
particles. 


Heuristic  Evaluation 

Usability  testing  is  critical  to  the  development  of  an  effective  software  application  to  ensure  that  the 
learner  is  able  to  focus  on  the  educational  objective  rather  than  the  process  of  completing  tasks  in  the 
virtual  environment.  The  heuristic  usability  evaluation  is  a  systematic  approach  to  software  development 
that  is  designed  to  identify  significant  usability  issues  early  in  the  software  development  life  cycle.  The 
heuristic  evaluation  of  this  application  was  modeled  on  the  method  proposed  by  Nielsen  (1993)  for 
software  usability  testing.  Each  new  evaluator  contributes  progressively  less  new  information  regarding 
usability  issues  and  problems.  By  the  fifth  usability  tester,  85%  of  problems  have  been  identified,  and 
additional  testers  mostly  replicate  information  that  has  already  been  discovered.  Tang  and  colleagues 
(Tang,  2006)  found  that  a  second  iteration  of  the  heuristic  methodology  demonstrated  additional 
improvement  in  the  user  interface  that  was  being  developed. 

In  this  study,  a  convenience  sample  of  medical  students  and  one  faculty  member  was  recruited  by  email. 
Two  sessions  were  planned  separated  by  approximately  one  month  of  software  development.  Open- 
ended  comments  by  the  evaluators  were  collated  and  categorized  as  gaming  or  content  issues. 


Individual  User  Evaluation 


A  convenience  sample  of  medical  students  was  recruited  by  email.  Users  were  given  a  group  of  seven 
tasks  to  perform  before  completing  a  survey.  The  survey  consisted  of  five  domains,  each  rated  on  a  7- 
point  Likert  scale  from  1  unacceptable  to  7=exceeds  expectations.  An  acceptable  level  was  defined  as  a 
score  >  4.  The  domains  were  learnability,  efficiency,  memorability,  errors,  and  satisfaction.  The  task  list 
for  users  was  this: 

1 .  Login  to  the  virtual  nephron  game; 

2.  Orient  to  the  environment:  within  the  first  person  view,  change  the  orientation  to  identify 
proximal,  distal,  cortical,  and  medullary  perspectives; 

3.  Use  the  paddle  interface  to  find  and  select  the  glomerulus,  proximal  convoluted  tubule,  and 
collecting  duct. 

4.  Use  the  paddle  interface  to  change  the  view  from  hovering  view  to  head-up  view,  and  from 
hovering  view  to  foot  view; 

5.  Use  the  paddle  interface  to  change  the  functional  representation  of  the  nephron,  and  to  activate 
sodium  particles  and  urea  particles; 

6.  Use  the  paddle  interface  to  navigate  to  different  sites  of  active  transport  (depicted  in  the 
mechanical  view  as  moving  pistons; 

7.  Exit  the  virtual  environment. 


Page  32  of  86 


Figure  4:  A  problem-based  learning  session  using  the  nephron  simulation  and  a  3D  rear-mounted 
projector.  A  single  medical  student  controls  the  simulation  using  a  handheld  device.  The  session  was 
mediated  by  a  faculty  member. 


Data  Analysis 

Before  and  after  survey  data  were  analyzed  using  a  paired,  t-tailed  Student’s  t-test.  Descriptive  statistics 
are  presented  next. 


RESULTS 

Group  User  Evaluation 


Sixty-one  students  (98%  of  the  class)  participated  in  six  sessions  with  the  systenm.  Eighteen  percent 
frequently  used  videogames,  and  16%  frequently  used  a  Nintendo  Wii.  Seventy-two  percent  stated  that 
they  had  used  3D  glasses  in  a  movie  theater,  but  only  32%  had  used  an  educational  game  in  medical 
school. 

The  motion-tracking  interface  using  the  head  mounted  tracker  with  a  hand-held,  paddle-mounted 
tracker  proved  to  be  problematic  for  members  of  the  audience  who  were  also  watching  the  projected 
images.  Observers  became  disoriented  while  the  device  was  being  used;  this  problem  was  resolved  by 
eliminating  the  head-mounted  motion  tracker. 

Students  reported  that  some  of  the  particles  that  were  modeled  were  difficult  to  see.  Modeling  of 
particle  dynamics  in  the  proximal  convoluted  tubule  was  incomplete  and  subject  to  misinterpretation. 
Other  content  and  usability  issues  were  recorded  and  provided  to  software  programmers  for  further 


Page  33  of  86 


development.  For  example,  one  task  was  to  increase  the  size  of  the  white  particles  (simulating  sodium 
ions)  to  make  them  more  visible. 


(A)  Self  Efficacy  <B>  Good  Mental  Image 


Figure  5:  Before/after  group  results,  n=61,  l=strongly  disagree  to  5=strongly  agree.  Vertical  and 
horizontal  bars  are  95%  confidence  intervals.  (A)  “I  feel  confident  in  my  understanding  of  nephron 
physiology.  ”  (B)  “I  have  a  good  mental  image  of  renal  physiology.  ”  (C)  'Renal  physiology  is  hard  to 
visualize”  (D)  Individual  usability  study,  n=5,  1  ^unacceptable  to  7=exceeds  expectations. 

Students  agreed  that  the  exercise  gave  them  a  good  mental  image  of  nephron  physiology,  with  a  mean 
rating  increasing  from  2.90  to  3.75  on  a  5-point  Likert  scale  (l=strongly  disagree  to  5=strongly  agree),  p 
<  0.01  (Figure  5 A).  They  disagreed  that  nephron  physiology  was  hard  to  visualize,  with  a  mean  rating 
changing  from  3.38  to  2.75  on  the  Likert  scale,  p  <  0.01  (Figure  5B).  Students  agreed  that  their  self- 
confidence  improved  as  a  result  of  the  exercise,  with  the  mean  rating  increasing  from  2.76  to  3.36,  p  < 
0.01  (Figure  5C).  Overall,  students  found  the  game  interesting  (80%)  and  thought  that  it  should  be  made 
a  regular  part  of  the  curriculum  (70%)  (Figure  5D). 

Heuristic  Evaluation 

Four  medical  students  and  one  computer  science  faculty  member  completed  the  evaluation.  In  the  initial 
session,  gaming/interface  issues  constituted  64%  (36/56)  of  the  open-ended  comments.  This  increased  to 
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93%  (50/54)  of  the  open-ended  comments  during  the  second  session,  suggesting  that  important  content 
issues  had  been  addressed  in  the  revised  version  of  the  software. 

Individual  User  Evaluation 

Five  medical  students  participated  in  one  user  evaluation  session  each.  All  domains  were  rated  at  a  level 
that,  “exceeds  expectations”  on  the  Likert  scale  (Figure  5).  This  suggested  that  the  iterative  approach  to 
software  development  was  successful. 


FUTURE  TRENDS 

We  believe  that  simulations  will  provide  an  increasingly  realistic  and  effective  method  of  teaching 
concepts  to  medical/healthcare  students.  Human  physiology  is  very  complex,  and  still  not  completely 
understood  (Thomas  et  al.,  2006).  Creating  models  and  performing  simulations  has  the  potential  to  help 
us  gain  a  more  complete  understanding  of  how  the  body  works,  and  ultimately  improve  patient  care.. 

The  modeling  of  the  physiologic  variables,  as  well  as  the  addition  of  stylistic  components  of  gaming 
motifs,  remains  a  work  in  progress.  Based  on  feedback  from  the  students,  the  current  version  of  the 
nephron  game  is  probably  most  suitable  for  novice  learners  of  renal  physiology.  Although  users  felt  that 
the  interface  was  easy  to  learn,  the  present  software  is  probably  optimally  used  with  a  knowledgeable 
technical  facilitator  present.  The  development  of  the  virtual  reality  application  for  use  in  front  of  a  large 
3D  screen  makes  the  system  amenable  to  greater  integration  into  a  problem-based  learning  curriculum 
that  emphasizes  small  group  interaction  and  formative  assessment  facilitated  by  a  faculty  member. 

Our  future  plans  are  to  integrate  the  kidney  simulation  with  a  more  dynamic  game  engine.  For 
example,  by  using  Monte  Carlo  simulation,  we  could  extend  the  simulation  using  random  variables  and 
statistics  to  predict  how  different  states  of  kidney  respond  to  changing  physiologic  conditions  (Roberts, 
1992).  Additionally,  future  studies  would  be  valuable  to  assess  the  contribution  of  gaming  elements  to 
student  learning  and  satisfaction,  as  well  as  to  compare  the  impact  of  immersion  in  this  virtual  system  to 
traditional  non-immersive  forms  of  education. 


CONCLUSION 

We  have  presented  a  working  prototype  of  a  simulation  of  renal  physiology  for  use  in  research  and 
teaching  medical  students  about  the  kidney.  Using  Flatland  (2008),  we  model  the  kidney  particles 
moving  in  a  virtual  3D  simulation.  The  usability  study  results  show  that  a  multi-agent  simulation  of 
kidney  function  system  can  improve  the  learning  progress  to  the  medical  students. 

The  structure  and  implementation  of  multi-agent  systems  in  this  project  also  have  the  potential  to  help 
healthcare  students  understand  the  interaction  between  particles  and  different  parts  of  nephron  using  a  3D 
virtual  reality  simulation  of  kidney  function. 
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Immersive  Three-dimensional  Virtual  Reality 
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Abstract 

Objectives:  Virtual  reality  (VR)  environments  offer  potential  advantages  over  traditional  paper  methods, 
manikin  simulation,  and  live  drills  for  mass  casualty  training  and  assessment.  The  authors  measured  the 
acquisition  of  triage  skills  by  novice  learners  after  exposing  them  to  three  sequential  scenarios  (A,  B, 
and  C)  of  five  simulated  patients  each  in  a  fully  immersed  three-dimensional  VR  environment.  The 
hypothesis  was  that  learners  would  improve  in  speed,  accuracy,  and  self-efficacy. 

Methods:  Twenty-four  medical  students  were  taught  principles  of  mass  casualty  triage  using  three  short 
podcasts,  followed  by  an  immersive  VR  exercise  in  which  learners  donned  a  head-mounted  display 
(HMD)  and  three  motion  tracking  sensors,  one  for  their  head  and  one  for  each  hand.  They  used  a  ges¬ 
ture-based  command  system  to  interact  with  multiple  VR  casualties.  For  triage  score,  one  point  was 
awarded  for  each  correctly  identified  main  problem,  required  intervention,  and  triage  category.  For 
intervention  score,  one  point  was  awarded  for  each  correct  VR  intervention.  Scores  were  analyzed  using 
one-way  analysis  of  variance  (ANOVA)  for  each  student.  Before  and  after  surveys  were  used  to  measure 
self-efficacy  and  reaction  to  the  training. 

Results:  Four  students  were  excluded  from  analysis  due  to  participation  in  a  recent  triage  research  pro¬ 
gram.  Results  from  20  students  were  analyzed.  Triage  scores  and  intervention  scores  improved  signifi¬ 
cantly  during  Scenario  B  (p  <  0.001).  Time  to  complete  each  scenario  decreased  significantly  from  A 
(8:10  minutes)  to  B  (5:14  minutes;  p  <  0.001)  and  from  B  to  C  (3:58  minutes;  p  <  0.001).  Self-efficacy 
improved  significantly  in  the  areas  of  prioritizing  treatment,  prioritizing  resources,  identifying  high-risk 
patients,  and  beliefs  about  learning  to  be  an  effective  first  responder. 

Conclusions:  Novice  learners  demonstrated  improved  triage  and  intervention  scores,  speed,  and  self- 
efficacy  during  an  iterative,  fully  immersed  VR  triage  experience. 

ACADEMIC  EMERGENCY  MEDICINE  2008;  15:1-6  ©  2008  by  the  Society  for  Academic  Emergency 
Medicine 

Keywords:  educational  technology,  immersive  virtual  reality,  task  performance  and  analysis, 
triage,  patient  simulation 


Technology  enabled  learning  systems  (TELS)  com¬ 
prise  a  spectrum  of  digital  learning  activities  that 
have  the  potential  to  be  effective  and  efficient  in 
training  first  responders  to  respond  to  mass  casualty 
incidents.1  TELS  incorporate  traditional  computer-based 
training  activities,  but  also  include  unusual  technologies 
such  as  immersive  virtual  reality  (VR).  TELS  are  appeal¬ 
ing  as  educational  tools  for  first  responder  training 
because  they  can  be  tailored  to  unique  situations,  pro¬ 
vide  engaging  learning  experiences,  and  be  reused.2,3  In 
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2005,  only  14  of  198  federal  training  courses  on  terrorism 
utilized  TELS;  only  31%  were  simulation-based,  and  only 
6%  used  VR  systems.2 

Because  of  the  paucity  of  VR  systems,  few  studies 
have  measured  learning  effects  in  VR  environments.  In 
one  recent  study,  the  knowledge  structure  of  medical 
students  triaging  a  single  head  trauma  patient  showed 
significantly  higher  gain  using  a  fully  immersed  VR  sys¬ 
tem  with  a  head-mounted  display  (HMD)  compared  to 
using  a  partially  immersed  (computer  screen)  system.4 
We  wanted  to  explore  how  novice  learners  would  per¬ 
form  when  encountering  multiple  simulated  casualties 
in  a  fully  immersed  VR  environment.  Do  they  become 
faster  and  more  accurate  in  triaging  the  patients  after 
practice  and  feedback?  Do  they  improve  in  their  appli¬ 
cation  of  lifesaving  interventions?  Additionally,  we 
wanted  to  determine  whether  their  perceived  self- 
confidence  regarding  first  responder  skills  was  affected 
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by  the  VR  experience.  To  conduct  the  study,  we  used  a 
novel,  fully  immersive  VR  environment,  in  which  the 
learners  wore  a  HMD  and  interacted  with  multiple  sim¬ 
ulated  casualties  using  a  gesture-command  system 
rather  than  a  computer  joystick  or  mouse.  The  study 
hypothesis  was  that  learners  would  improve  in  speed, 
accuracy,  and  self-efficacy. 

METHODS 

Study  Design 

This  was  a  repeated-measures  model  of  task  completion 
in  a  VR  environment,  with  subjects  serving  as  their 
own  controls.  The  study  flow  is  shown  in  Figure  1.  The 
University  of  Hawaii  Committee  on  Human  Studies  and 
the  United  States  Army  Medical  Research  and  Materiel 
Command  Human  Subjects  Research  Review  Board 
approved  the  research  protocol.  All  participants  signed 
written  informed  consent. 

Study  Setting  and  Population 

The  study  took  place  in  a  medical  school  simulation 
center  and  used  a  convenience  sample  of  medical  stu¬ 
dent  volunteers  who  were  recruited  by  a  single  e-mail 
announcement.  Mass  casualty  triage  training  is  not  part 
of  the  medical  school  curriculum  or  clinical  rotations. 
Twenty-four  subjects  were  recruited  and  enrolled. 

Study  Protocol 

Students  were  required  to  achieve  a  baseline  level  of 
triage  knowledge  before  they  could  begin  the  hands-on 
portion  of  the  study.  This  consisted  of  listening  to  and 
viewing  three  instructional  podcasts  for  a  total  of 
15:48  minutes,  followed  by  a  20-question  graded  exam. 
A  score  of  equal  or  greater  than  85%  correct  answers 
was  required  to  continue. 

During  the  VR  portion  of  the  study,  one  investigator 
ran  the  VR  computer  program  and  made  sure  that  the 
subjects  did  not  trip  while  wearing  the  HMD.  Learners 
received  a  scripted  orientation  to  the  VR  program  and 
then  spent  about  5  minutes  practicing  the  gesture- 
based  commands  in  a  test  scene.  After  completing  each 
VR  triage  scenario,  the  HMD  was  removed  and  learners 
watched  a  short  DVD  movie  (iDVD,  2007,  Apple  Inc., 
Cupertino  CA)  that  demonstrated  a  standardized  expert 
approach  to  triage  within  the  VR  scene. 

Technology.  The  VR  scene  was  created  using  Flatland 
(University  of  New  Mexico,  Albuquerque  NM),5  an 


open-source  information  visualization  system  that  runs 
on  Linux  OS  (http://www.linux.org/).  Visual  and  audio 
content  were  rendered  in  active  stereo  mode.  Users 
wore  a  Fifth  Dimension  Technologies  (5DT,  Inc.,  Irvine 
CA)  HMD,  stereo  earphones,  and  three  motion  tracking 
sensors  (Ascension  Technology  Corporation,  Burling¬ 
ton  VT),  one  for  the  head  and  one  for  each  hand  (Fig¬ 
ure  2).  Each  triage  scenario  consisted  of  five  casualties 
with  various  injuries  situated  in  a  dark  room.  Although 
auditory  and  visual  distracters  are  possible  in  the  VR 
scene,  such  as  police  sirens,  helicopters  with  lights,  and 
alarms,  the  distracters  were  turned  off  for  this  exercise. 
Users  examined  VR  patients  (Figure  3)  and  engaged  vir¬ 
tual  instruments  and  supplies  by  using  a  pose-  and  ges¬ 
ture-based  command  system.  For  example,  when  a  user 
raised  his  or  her  left  hand  overhead,  a  virtual  equip¬ 
ment  tray  appeared,  and  the  subject  could  then  "pick 
up"  a  virtual  instrument  using  the  right  hand  and  could 
then  use  the  virtual  instrument  or  perform  an  inter¬ 
vention  on  a  VR  casualty.  VR  casualties  were  not 
programmed  to  respond  to  interventions.  After 


Figure  2.  A  medical  student,  wearing  a  head-mounted  display 
(HMD)  and  two  sensor  gloves  checks  the  pulse  of  a  simulated 
casualty  in  the  virtual  environment. 


Figure  1.  Study  design.  RQ  =  reaction  questionnaire; 
learner  evaluation  questionnaire. 


Figure  3.  The  same  student  checks  a  simulated  casualty's  caro¬ 
tid  pulse  in  the  virtual  environment,  using  a  virtual  hand  that  is 
mapped  to  the  right  sensor  glove.  The  fingers  of  the  virtual 
hand  move  up  and  down  at  the  pulse  rate  when  positioned 
over  the  carotid,  radial,  or  femoral  regions.  Pulse  rate  and 
strength  also  appear  in  the  left  upper  visual  field  of  the  head- 
LEQ  =  mounted  display  (HMD).  Some  students  described  "feeling"  the 
Page  40p£f|§g  0f  th e  \/R  patient. 
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completing  an  intervention,  subjects  assigned  each  VR 
casualty  to  a  triage  category  using  a  four-color  triage 
tag  within  the  VR  environment.  They  also  selected  the 
main  problem  and  main  required  intervention  from  a 
pick  list.  The  gesture-based  command  system  has  been 
described  in  greater  detail  elsewhere.6  When  triage 
was  completed,  subjects  were  automatically  transported 
to  the  next  simulated  casualty  without  needing  to  navi¬ 
gate  within  the  VR  scene.  A  short  video  of  a  subject  tri¬ 
aging  five  VR  casualties  may  be  viewed  at  http:// 
bloodgoesroundandround.com. 

Scenarios.  Three  scenarios  consisting  of  five  adult 
casualties  were  created  in  the  VR  world.  Each  scenario 
consisted  of  three  "immediate"  patients,  one  "minimal" 
patient,  and  one  "delayed  or  expectant"  patient.  The 
immediate  injuries  were  one  hemorrhagic  shock,  one 
tension  pneumothorax,  and  one  airway  management 
problem.  Delayed  casualties  included  a  patient  with  a 
leg  fracture  and  a  patient  with  blunt  abdominal  trauma. 
One  "expectant"  casualty  had  massive  head  trauma  and 
anisocoria.  Minimal  patients  had  minor  wounds  with 
normal  vital  signs.  Learners  were  required  to  identify 
the  main  abnormality  (one  or  none),  perform  an  inter¬ 
vention  (one  or  none),  and  place  each  casualty  into  the 
appropriate  triage  category.  Main  abnormalities  fell 
into  these  categories:  airway,  breathing,  circulation, 
neurologic,  and  "other"  (such  as  fracture  or  psycholog¬ 
ical  injury).  Intervention  options  included  applying  a 
tourniquet,  using  a  HemCon  (HemCon  Medical  Tech¬ 
nologies,  Inc.,  Portland  OR)  bandage,  applying  a  regu¬ 
lar  bandage,  performing  a  needle  chest  decompression, 
and  inserting  a  nasopharyngeal  airway.  In  some 
instances,  "no  intervention"  was  the  appropriate 
response. 

Outcomes 

Three  outcomes  were  measured  for  each  scenario:  tri¬ 
age  score,  intervention  score,  and  time  to  triage.  For 
triage  score,  1  point  was  given  for  each  correct  answer 
that  was  selected  by  the  learner  in  the  VR  environment: 

1)  was  the  main  problem  correctly  identified,  2)  was  the 
required  intervention  correctly  identified,  and  3)  was 
the  triage  category  correctly  identified?  Thus,  each  lear¬ 
ner  could  receive  a  maximum  of  15  points  per  scenario. 
For  intervention  score,  1  point  was  awarded  for  each 
intervention  that  was  performed  correctly  in  the  VR 
environment.  Thus,  each  learner  could  receive  a  maxi¬ 
mum  of  5  points  per  scenario. 

Learner  Satisfaction  and  Self-efficacy.  Subjects  com¬ 
pleted  a  reaction  questionnaire7  (RQ)  for  the  completed 
VR  experience.  The  RQ  is  an  instrument  that  has  been 
used  to  assess  learner  satisfaction  with  Web-based 
training  material.  The  questionnaire  was  adapted  to 
assess  the  relevance  of  the  training  to  the  learner's 
perceived  role  as  a  first  responder  rather  than  to  the 
learner's  usual  clinical  role.  Subjects  completed  a  self- 
efficacy  questionnaire  before  the  VR  experience,  after 
Scenario  A,  and  after  Scenario  C.  The  questionnaire 
was  modeled  after  the  learner  evaluation  questionnaire 
(LEQ),  an  instrument  designed  to  measure  medical 
student  attitudes  toward  curriculum.8  We  modifi’e®^41 


subscale  of  the  LEQ,  the  self-efficacy  scale.  The  ques¬ 
tions  were  changed  to  take  the  viewpoint  of  a  first 
responder  instead  of  a  medical  student.  Two  additional 
self-confidence  questions  were  included.  Each  question 
was  scored  on  a  5-point  Likert  scale  with  points  labeled 
"never"  (1)  to  "always"  (5;  Table  1). 

Data  Analysis 

Learners  served  as  their  own  controls.  The  design  was 
a  repeated-measures  model  comparing  task  completion 
between  scenarios.  Previous  simulation  research  with 
manikins9  demonstrated  that  task  completion  was 
improved  most  between  the  baseline  and  subsequent 
scenario,  with  a  difference  of  about  30%.  Thus,  for  a 
correlation  of  r  =  0.80,  20  subjects  were  required  (tested 
against  a  constant  correlation  of  r  =  0.50,  /?  =  0.80, 
a  =  0.50,  tails  =  2;  source  of  estimation,  SPSS  Sample 
Power,  SPSS  Inc.,  Chicago  IL).  Results  are  reported  as 
mean  ±  standard  deviation  (SD).  Times  were  compared 
between  the  three  scenarios  using  one-way  analysis  of 
variance  (ANOVA)  for  each  student.  Triage  score  and 
intervention  score  were  analyzed  using  one-way 
ANOVA  for  each  student.  Self-efficacy  questions  were 
analyzed  using  one-way  ANOVA  for  each  student.  Post 
hoc  analysis  was  performed  using  Scheffe's  correction. 

RESULTS _ 

Demographics 

Twenty-four  students  scored  >85%  on  the  didactic  test 
and  completed  the  VR  exercise.  Four  of  the  students 
were  subsequently  excluded  from  the  analysis  because 
they  had  participated  in  a  manikin-based  triage 
research  project  within  6  months  that  used  similar 


Table  1 
Self-efficacy 


Before 

After 

p  Value 

1.  1  feel  confident  that 

1  will  learn  to  be  an 
effective  first  responder. 

4.0  (0.69) 

4.2  (0.62) 

0.034 

2.  1  feel  confident  that 
patients  will  consider 
me  an  effective  first 
responder. 

3.8  (0.64) 

4.1  (0.60) 

0.006 

3.  1  feel  confident  in 
my  ability  to  prioritize 
the  treatment  of  patients 
in  a  mass  casualty 
situation. 

3.2  (0.89) 

4.2  (0.52) 

0.001 

4.  1  feel  confident  in 
my  ability  to  prioritize 
the  use  of  resources  in  a 
mass  casualty  situation. 

3.1  (0.97) 

4.2  (0.75) 

0.001 

5.  1  feel  confident  in 
my  ability  to  identify 
high  risk  patients  for 
immediate  treatment  in 
a  mass  casualty 

situation. 

3.4  (0.88) 

4.2  (0.77) 

0.008 

Data  are  reported  as  mean  (±SD). 

1  =  never  to  5  =  always. 

>f  86 

5-point  Likert  scale: 

4 


Vincent  et  al.  •  IMMERSIVE  VIRTUAL  REALITY  TRIAGE  TRAINING 


triage  scenarios.  Most  learners  were  in  their  first  year 
of  medical  school  (MSI;  12/20;  60%),  but  the  study  also 
included  3  MS2  (15%),  3  MS3  (15%),  and  2  MS4  (10%) 
learners. 

All  five  self-efficacy  questions  showed  a  statistically 
significant  increase  in  scores  over  time.  Students 
became  more  confident  that  their  patients  would  con¬ 
sider  them  effective  first  responders  (p  =  0.006),  more 
confident  in  prioritizing  treatment  (p  =  0.001),  more  con¬ 
fident  in  prioritizing  resources  (p  =  0.001),  and  more 
confident  in  identifying  high-risk  patients  (p  =  0.008). 
Students  expressed  initial  confidence  that  they  would  be 
able  to  learn  how  to  be  an  effective  first  responder,  and 
this  score  also  increased  (p  =  0.034;  Table  1). 

Triage  score  improved  significantly  from  Scenario  A 
to  Scenario  B,  but  not  from  Scenario  B  to  Scenario  C 
(Figure  4).  The  26%  improvement  from  A  to  B  was  felt 
to  be  clinically  significant.  The  intervention  score  also 
improved  significantly  from  Scenario  A  to  Scenario  B, 
but  not  from  Scenario  B  to  C  (Figure  5).  The  average 
improvement  between  Scenario  A  and  Scenario  B  of 
more  than  one  correct  intervention  for  five  simulated 
casualties  was  felt  to  be  clinically  significant.  Time  to 
triage  improved  from  Scenario  A  to  Scenario  B  and 
from  Scenario  B  to  Scenario  C  (Figure  6).  The  overall 
improvement  by  4:12  minutes  is  likely  to  be  clinically 
significant. 

VR  Training  Evaluation 

The  students  rated  the  simulation  phase  of  the  course 
highly  (Table  2).  They  also  felt  that  the  pace  was  just 
right  (4.2  ±  0.39  on  7-point  Likert  scale,  1  =  too  slow 
and  7  =  too  fast).  The  level  of  difficulty  was  also  rated 
as  good  (4.5  ±  0.83  on  7-point  Likert  scale,  1  =  too  easy 
and  7  =  too  hard).  The  students  also  agreed  that  the 
course  was  relevant  to  them  as  health  care  providers 
(6.5  ±  0.61  on  7-point  Likert  scale,  1  =  strongly  disagree, 
7  =  strongly  agree). 


ABC 

Scenario 

Figure  4.  Average  triage  score  per  learner  (maximum  =  15). 
Lines  and  numbers  in  bars  are  standard  deviations  (SDs). 
*p  <  0.001.  y- Axis  label  =  triage  score.  Page 


Scenario 

Figure  5.  Average  intervention  score  per  learner  (maxi¬ 
mum  =  5).  Lines  and  numbers  in  bars  are  standard  deviations 
(SDs).  *p  <  0.001.  y- Axis  label  =  intervention  score. 


Figure  6.  Time  to  triage  one  scenario  consisting  of  five  simu¬ 
lated  patients.  Line  and  numbers  in  the  bars  represent  standard 
deviations  (SDs).  *p  <  0.001,  **p  <  0.05.  y- Axis  label  =  time  to 
triage. 

DISCUSSION 

We  showed  that  novice  learners  improved  their  perfor¬ 
mance  in  the  VR  environment  after  two  iterations  of  VR 
training,  with  triage  and  intervention  scores  improving 
significantly  during  the  second  scenario.  Learners 
worked  significantly  faster  with  each  new  set  of  VR 
patients,  with  time  to  triage  improving  significantly 
with  each  additional  scenario.  Learners  also  reported 
increasing  levels  of  self-confidence  throughout  the 
exercise.  Students  reported  that  they  were  more  confi- 
in  prioritizing  treatment,  prioritizing  resources. 
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Table  2 

Evaluation  of  VR  Triage  Course 


VR  Simulation 

1.  The  material  covered  was  relevant  to  my 
duties  as  a  health  care  team  member. 

6.5  (0.61) 

2.  The  course  objectives  were  adequately 
explained. 

6.3  (0.97) 

3.  The  course  was  well  organized. 

6.5  (0.69) 

4.  The  material  was  presented  in  an 
interesting  way. 

6.8  (0.55) 

5.  The  course  communicated  the  material 
effectively. 

6.7  (0.59) 

6.  As  the  course  progressed,  my  questions 
were  answered. 

6.8  (0.44) 

Data  are  reported  as  mean  (±SD).  7-point 

Likert  scale: 

1  =  strongly  disagree,  4  =  neutral,  7  =  strongly  agree. 

VR  =  virtual  reality. 

and  identifying  high-risk  patients  for  treatment.  They 
were  also  more  confident  that  their  patients  would  con¬ 
sider  them  an  effective  first  responder.  Self-efficacy  has 
been  shown  to  influence  learner  performance10  and 
may  also  predict  performance  on  objective  structured 
clinical  examinations.11 

Our  application  is  unique  in  its  combined  use  of  fully 
immersive  VR,  a  gesture-based  command  system,  and 
multiple  simulated  patients.  Other  computer-based 
training  systems,  such  as  the  Virtual  Medical  Trainer,12 
MediSim,13  and  Sim-Patient14  are  partially  immersive, 
menu-driven  VR  applications  that  display  the  simulation 
on  a  computer  screen.  BioSimMER15  was  an  immersive 
emergency  simulation  relating  to  a  biologic  attack  that 
used  a  HMD  and  simple  motion  gestures  for  navigation; 
development  was  discontinued  8  years  ago,  possibly 
due  to  the  complexity  and  high  cost  of  the  technology 
in  1999.  The  JUST  VR16  system  was  designed  to  simu¬ 
late  training  under  stressful  conditions  and  uses  a  rear 
a  projection  system  in  which  a  virtual  assistant  helps 
the  user  with  navigation  and  tasks  using  natural  lan¬ 
guage.  Project  TOUCH  is  an  immersive  VR  application 
from  which  our  current  application  was  derived.  Learn¬ 
ers  wear  an  HMD  and  use  a  joystick  to  navigate  within 
a  VR  environment  occupied  by  a  single  patient  with 
head  trauma.  Interestingly,  TOUCH  investigators 
recently  demonstrated  that  fully  immersed  students  had 
a  higher  gain  in  knowledge  structure  than  students 
who  viewed  a  computer  screen  and  used  a  mouse  to 
diagnose  and  treat  the  simulated  patient.4 

LIMITATIONS _ 

Students  who  volunteered  may  have  been  more  techni¬ 
cally  sophisticated  than  nonvolunteers,  resulting  in  bet¬ 
ter  scores  with  additional  VR  exposure.  Twenty-five 
percent  of  the  subjects  were  students  in  their  third  or 
fourth  year  of  medical  school.  Due  to  their  clinical 
experience,  they  may  not  have  been  representative  of 
most  first  responders  and  may  have  biased  the  results 
in  favor  of  a  greater  training  effect. 

We  introduced  a  new  gesture-based  command  sys¬ 
tem  for  navigating  in  the  VR  environment.  Some  qfa1^43 


performance  gains  that  students  exhibited  may  have 
been  due  to  increasing  familiarity  with  the  human/com¬ 
puter  interface,  rather  than  improvement  in  their  triage 
knowledge  structure. 

We  do  not  know  the  degree  to  which  the  improve¬ 
ments  in  triage  and  intervention  scores,  time  to  triage, 
and  self-efficacy  in  this  immersive  VR  environment  cor¬ 
relate  with  other  traditional  methods  of  teaching  triage 
such  as  manikin-based  simulation.  We  also  do  not 
know  how  results  in  this  immersive  VR  environment 
would  compare  to  more  traditional  computer-based 
simulations  that  are  partially  or  nonimmersive.  In  future 
studies,  we  plan  to  compare  the  performance  of  learn¬ 
ers  in  a  fully  immersed  VR  environment  with  perfor¬ 
mance  during  a  manikin-based  simulation. 

CONCLUSIONS _ 

We  conclude  that  principles  of  mass  casualty  triage  can 
be  effectively  and  efficiently  taught  to  novice  learners 
using  an  immersive  three-dimensional  VR  training  envi¬ 
ronment.  Further  studies  would  be  valuable  to  determine 
how  immersive  VR  training  could  complement  or  aug¬ 
ment  current  mass  casualty  incident  training  methods. 
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Appendix  C 


Evaluating  a  Virtual  Reality 
Motor-Skills  Simulator 

Curtis  IKEHARA  aJ,  Christoph  ASCHWANDEN  b,  Lawrence  BURGESS  b, 
Kevin  MONTGOMERY  c,  Darren  K.  C.  MOKb 
“Department  of  Information  and  Computer  Sciences,  University  of  Hawaii 
b  Telehealth  Research  Institute,  John  A.  Burns  School  of  Medicine, 
University  of  Hawaii 

c  National  Biocomputation  Center,  Stanford  University 


Abstract.  Evaluation  was  performed  on  a  prototype  low-cost  virtual-reality  motor- 
skills  simulator  (VRMSS)  created  at  the  Telehealth  Research  Institute,  John  A.  Bums 
School  of  Medicine,  in  conjunction  with  the  National  Biocomputation  Center, 
Stanford  University.  The  VRMSS  is  specifically  designed  to  teach  baseline  fine- 
motor  skills  used  in  surgery  that  are  based  on  a  matrix  of  elemental  technical  skills  that 
comprise  the  tenets  of  surgical  technique.  Fifty-seven  participants  were  randomly 
assigned  to  one  of  three  groups  (VRMSS,  box  trainer  or  no  training).  After  training 
each  group  was  evaluated  using  the  LapSim  from  Surgical  Sciences.  The  VRMSS  and 
box  trainer  were  similar  in  performance,  but  significantly  better  than  the  no  training 
control  group.  The  VRMSS  has  significant  advantages  over  the  box  trainer,  in  that  the 
VRMSS  can  provide  scoring  on  several  parameters  of  the  task  without  the  need  of  an 
instmctor  and  the  VRMSS  is  approximately  1/1 6th  the  cost  of  the  Lapsim. 


Keywords,  virtual-reality  fine  motor  skills  simulator,  laparoscopic  surgical  trainer 


Introduction 

A  prototype  low-cost  virtual-reality  motor- skills  simulator  (VRMSS)1  was  created 
using  the  SPRING  platform2  as  the  underlying  development  tool  at  the  Telehealth 
Research  Institute,  John  A.  Bums  School  of  Medicine,  in  conjunction  with  the  National 
Biocomputation  Center,  Stanford  University.  The  VRMSS  was  specifically  designed 
to  teach  baseline  fine-motor  skills  used  in  surgery  that  are  based  on  a  matrix  of 
elemental  technical  skills  that  comprise  the  tenets  of  surgical  technique.3  Following 
initial  audio/video  based  instmction;  bead-like  objects  are  moved  to  holding  cups/target 
areas  within  a  non- threatening  abstract  3 -Dimensional  environment  in  laparoscopic, 
microscopic,  and  endoscopic  formats  (see  Figure  1).  The  use  of  haptics  (touch,  force- 
feedback)  provides  more  human-computer  interaction  and  realism  then  previously 
possible  for  personal-computer  applications.  Complexity  is  increased  or  decreased  by 
changing  the  following  factors:  requirement  for  non-dominant  or  bimanual  hand  use 
and  environmental  changes  (depth  of  field,  decreased  exposure,  and  obstacles).  This 


1  Corresponding  Author:  Dr.  Curtis  Ikehara,  University  of  Hawaii,  Department  of  Information  and 
Computer  Sciences,  1680  East-West  Road,  POST  317,  Honolulu,  Hawaii  96822,  USA;  E-mail: 
cikehara@hawaii.edu 
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evaluation  of  the  VRMSS  was  funded  by  the  Telemedicine  and  Advanced  Technology 
Research  Center. 


Figure  1.  The  VRMSS  laparoscopic  trainer  for  grasping,  lifting  &  grasping  and 
coordination  skills  is  shown  with  dual  force-feedback  controllers. 


1.  Tools  and  Methods 

Fifty-seven  participants  were  randomly  assigned  to  one  of  three  groups  to  conduct 
an  evaluation  study  of  the  VRMSS.  To  assess  fine-motor  skills  participants  performed 
the  basic  skills  of  grasping,  lift  &  grasping  and  coordination  on  the  Lap  Sim  from 
Surgical  Sciences4  (see  Figure  2).  Each  group  then  received  time-equivalent  training. 
The  first  group  trained  on  the  VRMSS.  The  second  group  trained  on  a  box  trainer  used 
for  laparoscopic  training  (see  Figure  3).  The  third,  control  group,  read  through  two 
chapters  of  online  curriculum  on  telemedicine5.  Participants  were  then  reassessed  on 
the  Lap  Sim  for  changes  in  fme-motor  surgical  skills. 
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Figure  2.  LapSim  from  Surgical  Sciences. 


2.  Results 

The  grasping  and  lifting  &  grasping  tasks  were  not  significantly  difficult  and  all  groups 
performed  well  on  those  tasks.  Figures  4  and  5  show  highlights  of  group  performance 
on  the  LapSim  coordination  task  after  training.  Group  performance  times  in  Figure  4 
show  a  significant  difference  between  the  groups  (mean  =  66.3  seconds,  F=3.423, 
p<0.04).  The  post  hoc  test  (Tukey)  showed  that  VRMSS  and  the  box  trainer  mean 
completion  times  were  not  significantly  different  and  the  control  group  had  a 
significantly  longer  performance  time  (p<0.044).  Figure  5  shows  the  pass  percentage 
for  the  groups.  The  VRMSS  and  box  trainer  groups  were  very  similar  and  both  training 
were  at  least  1.75  times  better  than  the  control  group. 
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Figure  3.  Box  trainer  used  for  laparoscopic  training. 


3.  Discussion 

Training  on  both  the  box  trainer  and  LapSim  have  been  previously  shown  to  positively 
impact  operating  room  performance,6’7  With  the  VRMSS  and  box  trainer  groups  having 
similar  results  on  the  LapSim  assessment,  it  is  likely  that  VRMSS  would  also  have  a 
positive  impact  on  outcomes  at  surgery.  However,  like  LapSim,  VRMSS  has 
significant  advantages  over  the  box  trainer,  in  that  the  VRMSS  can  provide  scoring  on 
several  parameters  without  the  need  of  an  instructor  during  student  skill  acquisition. 
Parameters  of  importance  scored  by  the  VRMSS  include:  time  to  complete  task, 
number  of  errors  and  economy  of  movements.8  Also,  the  VRMSS  is  approximately 
1/1 6th  the  cost  of  the  LapSim. 
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LapSim  -  PerformanceTime 


Figure  2.  Shown  is  the  mean  time  of  each  group  on  the  coordination  task  after  receiving  training. 


LapSim  -  Passes  the  Test 


Figure  3.  Shown  is  the  percent  of  participants  from  each  group  that  passed  the 
LapSim  on  the  coordination  task  after  receiving  training. 
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Manikin  Triage 
Virtual  Reality  Triage 
Virtual  Nephron 
LapSim  Experiment 
Tangible  User  Interface 
Augmented  Reality 


PODCASTS 


Triage  Podcasts 

■  Introduction  to  Manikin  Triage 

■  Principles  of  Mass  Casualty  Triage 

■  Triage  Challenge  I 

■  Triage  Challenge  II 


STUDY  DESIGN 


MANIKIN  TRIAGE 


■  21  students 
were  enrolled 

■  Most  were  2nd  year 
students 

■  All  were  novices 
(without  prior  triage 
training) 


■  MSI  ■  MS2  ■  MS3  ■  MS4 


12 

Number 


TAGGING  ACCURACY 


MANIKIN  TRIAGE 


■  3  correct  responses  per 
simulated  patient 


■  1st  100  ■  Next  100  ■  Next  100 


■  300  possible  correct 
answers  for  20  students 
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■  Students  were  more 
accurate  after  10 
simulated  patients 

p  <  0.001 
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INTERVENTION  ACCURACY 


MANIKIN  TRIAGE 


One  correct  intervention 


p  <  0.001 
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simulated  patient 
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MANIKIN  TRIAGE 


Time  to  triage  5  ■  lst  5  patients  ■  Next5  ■  Next5 

simulated  patients 

■  Students  got  faster  after 

10  simulated  patients  8^7 
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p  <  0.001 
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STUDY  DESIGN 


SPEED 

VR  TRIAGE 


VR  Triage 

^  ■  Minutes 

Study 


7:38 


VR  TRIAGE  SELF-EFFICACY 

I  FEEL  CONFIDENT  THAT  I  WILL  .  .  . 


Improved  Self-Efficacy  after  VR  Triage 

*My  patients  will  consider  me  an  effective  first  responder 

*Be  able  to  prioritize  the  treatment  of  patients  in  a  mass  casualty 
situation 

*Be  able  to  prioritize  the  use  of  resources  in  a  mass  casualty 
situation 

*Be  able  to  identify  high  risk  patients  for  immediate  treatment  in 
a  mass  casualty  situation 


*  statistically  significant  results 


USER  EVALUATION 


VR  Nephron  Experiment 

■  60  first  year  medical  students 

■  Cardiovascular  -  renal  unit 

■  6  facilitated  sessions  (tutorial  groups) 


45  minutes  each 


USER  INTERFACE 

Student  uses  a  paddle  to  select  objects  in  the  nephron  model 


FACILITATED  SESSION 

Student  navigates  to  locations  that  demonstrate  physiologic  principles 


■  Before 


■  After 


I  feel  confident  in  my  understanding 
of  renal  physiology  . . . 


40 


30 

20 

10 

0 


SD  D  N  A  SA 


SD=Strongly  Disagree,  D=Disagree,  N=Neutral,  A=Agree,  SA=Strongly  Agree 


■  Before 


■  After 


Nephron  physiology  is  hard 
for  me  to  visualize  . . . 


SD  D  N  A  SA 


SD=Strongly  Disagree,  D=Disagree,  N=Neutral,  A=Agree,  SA=Strongly  Agree 


■  Before 


■  After 


I  have  a  good  mental  picture 
of  nephron  physiology  . . . 
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20 

10 
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SD  D  N  A  SA 


SD=Strongly  Disagree,  D=Disagree,  N=Neutral,  A=Agree,  SA=Strongly  Agree 


■  After 


I  found  the  nephron  game  interesting  . . . 
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SD  D  N  A  SA 

SD=Strongly  Disagree,  D=Disagree,  N=Neutral,  A=Agree,  SA=Strongly  Agree 


■  After 


The  nephron  game  helped  me  learn 
nephron  physiology  . . . 
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SD  D  N  A  SA 


SD=Strongly  Disagree,  D=Disagree,  N=Neutral,  A=Agree,  SA=Strongly  Agree 


■  After 


The  nephron  game  should  become 
a  regular  part  of  the  curriculum  . . . 
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SD  D  N  A  SA 


SD=Strongly  Disagree,  D=Disagree,  N=Neutral,  A=Agree,  SA=Strongly  Agree 


LAP  SIM  EXPERIMENT 


60  learners:  randomly  assigned  to  3  groups 
Control  Group,  Lap  Trainer  group,  SAGES  group 
Gold  Standard:  LapSim  performance 


Outcomes  Measured  on  LapSim  Trainer 


TANGIBLE  USER  INTERFACE 

Fast  Geometry  Acquisition 


User  moves  sensor  over  physical  object  to  rapidly  obtain  crude  3D  VR  representation 


Smoothing  occurs  in  3D  VR  image  with  more  movement  of  the  sensor  over  the  physical 
object 


VR  acquisition  of  the  convex  landmark 


Sensor  is  used  to  acquire  and  register  a  convex  landmark:  manikin  umbilicus 


3D  VR  mesh  representation  of  the  physical  object 


3D  VR  mesh  is  deformed  to  take  on  shape  of  the  physical  object 


Augmented  Reality  Interface 
using  Motion  Tracking  and  3D  VR 


Motion  Tracking  Sensors  +  Manikin  +  VR  Software 


Tapping  on  Physical  Model  is  Mapped  to  VR  Software 


Tapping  represents  a  programmable  Gesture-Command 


VR  Command-Gestures  elicit  an  output:  "ouch" 


